Over half a century after being coined, the term Artificial Intelligence (AI) has evolved rapidly. A few years ago, everyone was excited about its potential; today fears about its misuse are dominant. As elections take place in our country, an alarm has been expressed about “AI Elections,” where the ordinary Indian voter has been deemed to be at the highest risk of electoral misinformation— so great and, at times, uncontainable in scale that it could lead to sections of the people being deceived and disenfranchised. 

So far nothing of the kind has happened. The one drama we witnessed about an alleged “deepfake” of Amit Shah turned out to involve no AI at all but merely skilful (and malicious) editing of various things he had indeed said, to make it look as if he had said something else.

Despite the fact that we have not seen our 2024 elections taken over by AI, it turns out it’s not for lack of trying. Divyendra Singh Jadoun — the helmsman of the revolutionary and robustly successful start-up Polymath Solutions, better known as The Indian Deepfaker — has made some shocking revelations. Widely renowned for using AI “to create Bollywood sequences and TV commercials,” Jadoun says, as The Washington Post reports, that “hundreds of politicians have been clamouring for his services, with more than half asking for ‘unethical’ things. Candidates asked him to fake audio of competitors making gaffes on the campaign trail or to superimpose challengers’ faces onto pornographic images. Some campaigns have requested low-quality fake videos of their own candidate, which could be released to cast doubt on any damning real videos that emerge during the election.” Bound by ethics, Jadoun’s firm denies requests intended to defame or deceive, but numerous other deepfakers oblige, potentially warping the political reality of the world’s largest democracy as it staggers to the polls. Perhaps we have escaped the worst in the four phases that have voted so far, but can a serious AI-generated drama be far behind? 

Earlier this year, at the Mathrubhumi Literary Festival in Thiruvananthapuram (of which I am the founding Patron), the organisers conducted an interesting experiment using AI. They interviewed me on the subject of AI use, I answered innocently enough, and then when the Festival took place, they showed the interview – except that the questions were being asked in the video not by an interviewer but by an AI replica of myself!

The folks at the Mathrubhumi had my consent (and they destroyed the raw material used after the show was broadcast), but the point they made by conducting this experiment was pretty scary. Jadoun told The Hindu that while earlier it took eight to ten days to create a believable face-swap deepfake video (as the clip had to be converted into frames, which required thousands of images), it can now all be done in merely two to three minutes or for a few rupees at best; the process becomes easier still when the person knows how to code. In other words, the availability of free AI tools — abundantly available on the internet — can allow anyone to create a deepfake video. Powered by accessible generative AI models, deepfakes are no longer messy cut-and-paste edits one dismisses at first glance, but are so realistic they could fool even experts.

placeholder
AI interview during Mathrubhumi Literary Festival

According to a 2024 analysis published in PNAS Nexus — an open-access research journal of the Oxford University Press — “today’s large language models (LLMs) [such as ChatGPT] are enhancing bots with new features and efficiency, imbuing them with a deceptive human-like persona.” In other words, the unification of generative AI and automated bots is scary, for generative AI can “make it easier to code bots, expand the reach of manipulated messages and survive longer online.”

No wonder, then, that the World Economic Forum in its 2024 survey found India ranked the highest in the risk of AI-enabled misinformation and disinformation globally. The misuse of AI in elections “could seriously destabilise the real and perceived legitimacy of newly elected governments, risking political unrest, violence and terrorism, and a longer-term erosion of democratic processes.”

Several democracies have already flagged the propensity of generative AI to meddle damagingly in the electoral process, as happened in Taiwan’s elections earlier this year. On election day, a Chinese Communist Party-affiliated group posted AI-generated audio of a prominent politician (who had dropped out of the Taiwanese election) seemingly endorsing another candidate. But the politician, Foxconn owner Terry Gou, had never made such an endorsement, and did not intend to support his China-backed rival. YouTube took down the fake audio, and as it happened, Lai Ching-te — the candidate whom the leadership of the Chinese Communist Party had opposed — won, signalling the limits of the AI effort to affect the results of the election. 

But the worrying thing is that it might have worked. A Microsoft Report from April 2024 — titled “Same targets, new playbooks” — warned of similar Chinese attempts to thwart elections in India, South Korea and the United States this year. “China’s increasing experimentation in augmenting memes, videos, and audio will likely continue — and may prove more effective down the line,” the Microsoft report argued. 

placeholder
Representational image | Photo: Getty images
Representational image | Photo: Getty images

So far, we have no evidence that it has happened. However, the low cost and easy availability of generative AI tools have made it possible for people and parties without state backing to engage in cutting-edge deception that rival campaigns that governments like China could conduct. From Moldova to South Africa, the United States to Bangladesh, the looming spectres of generative AI and deepfake technology have already made their worrisome presence felt. In Bangladesh, especially, such deceptions acquired a disgustingly sexist tint when deep-faked bikini-clad pictures were disseminated on social media of Rumeen Farhana, an opposition politician. Confronting sexual harassment on the internet, Farhana wondered whether, in such conservative nations like hers, such AI-enabled character assassinations might prevent women candidates from entering the political fray. In this regard, Farhana strongly feels that any technology, be it of destruction or distraction, is first used to target women, and to this norm AI and deepfake technology are no exception.

While AI is lowering the costs and easing the difficulty of producing election content, social media platforms, search engines and even video streaming services have created an environment conducive to the widespread dissemination of deep-faked clips. In its bid to compete with TikTok, YouTube came up with Shorts (much like Instagram did with Reels); monetised and highly successful, these short clips are reportedly awash with unlabelled deepfakes and voice clones. Then there’s the problem of X having supplanted Twitter, which entailed Elon Musk’s laying off thousands of employees, including those working on trust, safety and content moderation. The result is an enabling environment for fakery.

Though Divyendra Singh Jadoun’s The Indian Deepfaker is scrupulous about watermarking its content on the internet, other deepfakers are not. So it is that social media platforms and video streaming services — Instagram, X, Facebook and YouTube — are at a disadvantage when detecting anything more sophisticated than a low-quality deepfake. Jadoun himself recalls that when he tried uploading some of his firm’s deepfakes on Facebook and YouTube, they bypassed the sites’ security filters. Taken together, all these trends have allowed AI and deepfake technology to permeate every aspect of our lives, obfuscating public discourse and blurring the lines, especially during election season, between fact and fiction.

It’s clear that there is a need to find the right balance between technological development and the protection of our human and political rights. I will discuss some ideas in my next column on the subject.