Company co-founder, data-mining expert, author, certified yoga instructor, and life explorer – A/Prof Nitin Indurkhya is also teaching a range of subjects including data analytics at SIT. (SIT Photo: Keng Photography/Tan Eng Keng)

In Christopher Nolan’s superhero film The Dark Knight, Batman harnessed cutting-edge spyware technology to capture the villainous Joker by hacking every phone in Gotham City. Steven Spielberg’s science fiction flick Minority Report saw Tom Cruise’s character on the run when an advanced crime prediction software accused him of a future murder. 

Both movies have one thing in common. “Big data is right in the middle of it,” said Associate Professor Nitin Indurkhya, who specialises in the subject at the Singapore Institute of Technology (SIT). And Hollywood has given big data a bad rep. 

“There are a lot of movies that portray how big data is going to take over and dehumanise the world. You’d think that it’ll go haywire,” he said.

But in the real world, over three decades of experience in the field tells him otherwise. “Big data helps with quality of life, raise standards of living and improve productivity,” said A/Prof Indurkhya, who coined the term ‘big data’ while co-authoring a predictive data guidebook published in 1998. “There are a lot of benefits.”

This is thanks to a rapidly improving ability to process valuable information from the sheer quantity of digital information available worldwide today. In essence, this is what big data is all about. “It is all about scale. It’s referring to the amount of data we have,” he explained. 

And if this massive trove of information is applied correctly using techniques like predictive analytics and machine learning, the world only stands to benefit.

It is one of the many lessons that the academic, who joined SIT in July 2021, wants to impart to his students. “I’m actually very optimistic that human beings will find a way to coexist with machines and artificial intelligence (AI) in a very creative manner,” he said.

The Ultimate COVID-19 Weapon

The spotlight on big data has significantly brightened in the last two years when it became a powerful tool to battle the COVID-19 pandemic. For instance, the ability to access and analyse large data sets of patients worldwide allowed common symptoms to be quickly identified. This, in turn, helped facilitate faster diagnosis and treatment.

“If patients can get diagnosed really fast, it can mean the difference between life and death for them,” he said.

This dual force of information and prediction also helped countries take preemptive measures to prepare for incoming waves of cases. For instance, BlueDot Insights – a predictive analytics cloud platform – was crucial to Taiwan’s stellar success in containing the initial outbreak of the virus. Getting an accurate forecast of the swiftly emerging threat allowed the country to shut its borders early, saving lives in the process.  

“These are some of the ways in which predictive analytics can really help when you have something that is fast-moving, and you need information on how things evolve,” he said.

But one area where big data truly helped turn the tide against the pandemic was through the rapid development and rollout of vaccines. What typically took 10 to 15 years to make was achieved in months.

Big data was central to the lightning-fast success. “This happened because people were using predictive analytics techniques to do trials,” explained A/Prof Indurkhya. “By using computers to do simulations, they could use analytic techniques to fast-track the probe.”

A/Prof Indurkhya explains how big data plays an integral role in positively addressing complex problems, such as combatting COVID-19. (SIT Photo: Keng Photography/Tan Eng Keng)

Keeping Up with The Joneses

With speed characterising the big data space, he also has another key lesson for his students: the need to remain agile and adaptable.

“This field changes so rapidly,” he said. “They need to learn with the awareness that whatever they learn, it has a certain life cycle. After that, they have to move on to new things.”

Case in point: a decade ago, Hadoop – a software analytical framework – was all the rage. As an individual computer could not process large amounts of data then, Hadoop allowed data to be stored across multiple computers for simultaneous processing.

But today, advanced Graphics Processing Units and better distributed processing architectures means that Hadoop may now be less relevant. “In the future, something else might come up. We have to be ready and we’ve got to be flexible,” he said.

There is one final, and perhaps the most important lesson that A/Prof Indurkhya’s work in big data has surfaced: it might actually have the power to unite the world.

“By pooling data across different populations and cultures, finding so many regularities actually proves that we’re all one,” he said. “There is a humanity that is common in all of us.”