Author: Tarun Khanna

Researching the Maha Kumbh Mela — the religious festival that takes place every 12 years near Allahabad in north India, at which over 100 million Hindus gather — has more than its fair share of challenges. (And I’m not talking here about a tragic railway footbridge collapse or the incessant rain that flooded the tent city last week.) Setting up a live experiment is never easy and always exciting. Our goal: to trace ground-up the emergence of a market, and figure out the structures and mechanisms that will allow it to work more efficiently.

Consider the numerous choices that my colleagues, Columbia Business School’s Emily Breza and Microsoft Research’s Arun Chandrasekhar, and I faced when we decided to study how information about demand and supply circulates, and how prices evolve in the huge market that springs up during the Kumbh Mela. Our focus was on the numerous vendors who move in overnight to cater to the millions of people who attend the Mela by selling food grains, vegetables, fruits, and other provisions.

We first identified the main sources of demand. One is the approximately 500,000 pilgrims, called kalpvasis, who camp on the festival grounds for its entire duration. Another, probably the biggest, is itinerant pilgrims, who spend a day or two at the Mela. Government officials managing the event, such as bureaucrats, security personnel, and healthcare workers, constitute another source of demand. And observers like us, attending the event to report on it or for research (and, in my case, a little bit of spirituality) also need to buy groceries.

On the supply side, we had to decide which vendors to survey so we ended up with a representative sample. We asked ourselves: Is the process of setting prices different for each product? We assumed it is, so we had to create a sample accordingly. Other issues quickly arose: How many of each kind should we survey? Since we are interested in vendor as networks, should we choose vendors who clustered together or ensure they were spatially separated? And so on.

After making some assumptions, we had to pick a large number of vendors, explain how they could help us, and convince them that it would not waste their time. We then employed some surveyors, whom we trained in data collection. Their job was to ensure the accuracy of the data that the vendors generated in the necessary formats and frequency.

We wondered if we’d have to oversee the monitors, but as it turned out, several of them had already participated in field studies conducted by the Institute for Financial Management and Research, and knew what they were doing. Another colleague, Vikas Dimble, heroically managed the on-the-ground data gathering process, making our lives infinitely easier.

Tracking several other factors helped us cross-check the data’s validity and interpret it correctly. For example, each commodity commanded a different price on the Mela grounds, but Allahabad’s wholesale market was the main source for most of them. The variations between wholesale and retail prices had obviously something to teach us.

One goal is to identify the factors that drive price changes, so we had to be creative. For instance, if the price of a Mela staple, such as sweet potato, shot up, was it because of a breakdown in supplies due to say rain, or was it because of a demand surge caused by a sudden influx of pilgrims? To find out, we could artificially boost demand for a while by handing out small sums of money, or we could augment supplies by offering vendors small subsidies.

If some information was hard to come by — the wholesaler’s location, for example —-its impact on price was obviously of interest. So we highlighted where the wholesalers were located to a randomly chosen subset of retailers, which changed the amount of products bought and sold as well as prices.

Building the data-set and making interventions requires the establishment of trust between buyers, sellers, regulators, surveyors, and researchers. Once we’ve crunched the numbers, we’ll not only be able to observe a market as it grows, but identify what practices helped it work best.

Between 2000 and 2010, the population of Delhi burgeoned from 15 million to 22 million while Shanghai’s population swelled from 14 to 20 million. Compare that to the recent rise of an impromptu city near Allahabad in India: In the week after January 14, 2013, the first day of the Maha Kumbh Mela festival — during which Hindus gather for a sacred bath at the confluence of the Ganga and Yamuna rivers — around 10 million people had gathered there.

When the event ends five weeks later, approximately 100 million people would have moved into and out of Allahabad. (I say “approximately” because the precise numbers are difficult to come by.) It took 60 years for the population of Istanbul to grow from one to 10 million, and 50 years in the case of Lagos. At Allahabad, though, the population rose from zero to 10 million, give or take a few million, in just a week’s time.

That’s a slightly unfair comparison because the local government isn’t going to put in place all the fixtures of a functional metropolis. However, it’s only partly unfair. The Indian authorities do have to pull off the creation of a huge temporary tent city with minimal mishap. An enormous amount of urban planning, civil engineering, governance and adjudication, and maintenance of public goods — physical ones like toilets as well as intangibles such as law and order — and plans to deal with unexpected events goes into the creation of this city. Those are pretty much the main elements surrounding the creation of any city in the world.

There will also be a reasonably efficient dissolution of the city when the Kumbh Mela ends in late February, but that’s another story. Some cities have declined over time, but I can’t even imagine what it would take for one of the world’s major metropolises to unwind.

The mammoth people flows at Allahabad got me excited when two colleagues at Harvard University, religion professor Diana Eck and design professor Rahul Mehrotra, broached the idea of studying the Maha Kumbh Mela some months ago. As a child growing up in India, I had read about the festival, but had never entertained the idea of visiting it or studying it. Having lived outside India for over two decades, I now find myself in a position to revisit the event, intellectually and physically.

The flows of humanity that my colleagues and I will study during the five weeks of the Kumbh Mela will shed light on similar events, such as responses to unexpected events, disasters, and the like, that will take decades to unfold in other metropolises. Some researchers are social anthropologists, in effect, following key officials during the Mela to unmask the processes that allow efficient and rapid decision making. In a sense, the festival is a laboratory setting that scientists of all sorts constantly look for. While there are other large gatherings of folks, such as the Hajj pilgrimage to Mecca, those are a tenth of the size in terms of the number of participants.

Another issue of interest is the emergence of social structure in complex groupings. The Kumbh Mela authorities put down some bright lines on who gets to go where, when, and how — for example, rules that govern people’s movements during some religious days — and some rules are determined by long-standing customs. Other, more informal norms among disparate groups of people seem to emerge quickly. To those interested in how cooperation among diverse groups happens, this is a fortuitous setting.

This is also the first Big Data Kumbh, as I call it. With cellphone usage ubiquitous in India, the millions of cellphones at the Kumbh Mela will act as mobile sensors. My colleagues and I have undertaken, with the help of local cellular providers and government authorities, to amass, arguably, the biggest ever telecom data set.

To imagine the uses to which researchers could put the data, consider these hypothetical ideas. The data could be used to understand how untoward incidents have been contained. After all, the Maha Kumbh Mela has managed to prevent major disasters for a long time. Why don’t disasters spiral out of control when massive numbers of people, unfamiliar with each other, are involved? Can we spot the signatures of an incipient disaster in the data, and the process by which those signals are attenuated rather than amplified?

There is much commerce, as well as charitable exchange, of goods and services at the Kumbh Mela. How do vendors deal with the inevitable errors in forecasting demand? Do inter-vendor communication patterns allow the collective containment of uncertainties? Indeed, the telecom data generated at the Kumbh Mela should provide grist to the intellectual mills of statisticians, engineers, mathematicians, and social scientists for a long time, and allow us to model the use of this kind of Big Data.

We’ll report on our findings here over the coming weeks.