Abstract

In India, cyber related harms are primarily governed by the Information Technology Act, 2000, whereas personal data privacy and data processing related harms are regulated under the Digital Personal Data Protection Act, 2023. However, there is a growing need for the legislative architecture to explicitly contemplate the risks emerging from Generative AI, large model training, autonomous inference and non-state algorithmic manipulation. Generative AI fundamentally alters the locus of risk. The harm is no longer merely content driven, but model driven. The core legal question is no longer only “who posted” or “who processed” but “how was the model trained”, “what datasets fed it”, and “who bears constitutional responsibility if the model inherently embeds systemic privacy risk?”. AI is now generating complex new harms such as deepfakes used in child pornography, financial fraud, political manipulation, character assassination, online extortion and large-scale copyright violations, which neither the IT Act nor the DPDP Act were drafted to address. It is unclear to pin down the liability of AI driven harms and there is a clear legislative vacuum in dealing with such crimes. This paper critically evaluates whether the current legal framework is sufficient to safeguard informational privacy, autonomy and fundamental rights in India. It argues that significant regulatory gaps persist within the existing framework, particularly concerning AI training without consent, risk of re-identification of anonymized datasets, opacity in algorithmic decision-making, and accountability mechanisms for private AI developers. The paper concludes that while the IT Act and the DPDP Act constitute necessary baseline instruments, they are not sufficient to address the multidimensional harms posed by AI, necessitating an AI-specific legislative intervention that deals with its technical architecture. 

Digital harms in the current online ecosystem have become far more complex and technologically sophisticated than traditional cyber offences. Deepfakes now enable the fabrication of sexually explicit content using real faces and also facilitate large-scale political manipulation during elections by generating highly realistic false videos and statements. AI further creates serious copyright challenges because models are trained on massive datasets without consent, leading to unauthorized derivative outputs which undermine the rights of original creators. Parallelly, darknet markets continue to serve as hidden criminal networks for drug trafficking, arms trade, ransomware services and sale of stolen identity data, operating beyond conventional jurisdictional enforcement. Beyond these, newer AI-driven harms are emerging such as algorithm-based radicalization, AI generated personalized phishing, biometric identity cloning, synthetic voice impersonation, AI-enabled financial scams and targeted behavioral manipulation. 

Section 2(1)(w) of the IT Act defines "Intermediary" as any person who on behalf of another person receives, stores, transmits that electronic record or provides any service with respecty to that record. This usually makes Telecom service providers, Network service providers, Internet service providers, Search Engines, Online payment sites liable for any convention of this act. However, the AI systems don't just transmit the electronic record or information, rather they generate new information. Hence, they do not qualify under the definition of Intermediary. The IT Act have evolved in the Era where the digital data is largely conceived as "Static". Today, the data created by AI is dynamic based on user prompts and at times, modify their own outputs based on the use case.

Further, there are several actors involved in modern AI operations. There is a developer who builds the AI model, a deployer who implements such AI model in their product and the User who interacts with AI. The IT Act does not recognize these roles and hence the liability is misplaced. Section 79 of the IT Act provides safe harbor protection to the Intermediaries. With the evolving use of AI products by the intermediaries, such safe harbor protection cannot be availed by the provisions ascertained in Section 79.

Towards storage, protection and processing of personal data, India has enacted the Digital Personal Data Protection Act, 2023. The law is not yet in force and is awaited for further recommendations from various stakeholders for drafting the rules. However, serious legislative vacuum has be pointed out as the Statute fails to address the threats posed by the use of Artificial Intelligence by the data fiduciaries.

Section 17(2) of DPDP Act provides exemptions for storing Publicly Available Personal Data with the exemptions being for research, public security or any legitimate uses. Model AI models train their algorithms using publicly available text, images from the websites, other scrapped content from millions of users. Even if the data is public, the user did not consent to the use it as a training material for commercial use of AI models. It is unclear whether such massive scrapping is qualified as "legitimate use" where the data fiduciaries can claim the exemption. Further, even if the data fiduciary takes the consent from the user for the use of personal dara, such consent can be applied only for specific purpose. In reality, the AI training models needs the data for broad undefined future tasks also. Hence, the law remains silent on such use and does not provide users to opt out of such continued use.

In AI, data that is obtained for a specific purpose may be used to train multiple downstream models and in reality, Developers may not be able to fully predict the future uses at the time of collecting data. In short, AI training is purpose agnostic while the DPDP act requires consent for specific purpose making it structurally incompatible.

The DPDP Act treats AI developers same as any other ordinary data fiduciaries. The Act fails to recognize AI model developers, AI deployers, AI model trainers or downstream users. In other words, the entire AI life cycle is ignored and the hence the liability on the individual players becomes weak. The risks of various use cases of AI is not defined and hence the broad legislative language is not entirely helpful in resolving specific harms.

The DPDP Act does not impose liability of AI developers and they do not have duty to disclose the type of data sets that are used for training their models. Hence, it is not helpful in identifying if the model is trained through publically available scrapped material or copyrighted material. Further, there are no specific risk assessments, algorithmic audits or documentation of training practices provided as obligations for the developers or the deployers under the Act. Further, there are no safeguards for algorithmic fairness, discrimination or bias unlike other global AI governance models.

Today, the use of generative AI has triggered Copyright issues extensively. In Andersen v. Stability AI, Midjourney & DeviantArt (U.S., 2023), Sarah Silverman v. OpenAI & Meta (U.S., 2023), and The New York Times v. OpenAI & Microsoft (U.S., 2023), the legal vacuum in existing regulatory frameworks to determine if training generative AI models on copyrighted datasets qualifies as fair use or requires explicit consent remains unresolved.

Section 2(d)(v) of Indian Copyright Act defines author as in relation to any literary, dramatic, musical or artistic work which is computer-generated, the person who causes the work to be created. Here the law recognizes computer generated works and awards authorship to a human who caused the work through computer. The generative AI models are trained using vast datasets that are publically available. This gives rise to significant legal questions like who owns the authorship of such work- Developer or deployer or original author whose work was used to train the AI algorithm?

Further, under Section 52 of the Copyright Act, Research studies are given fair dealing exception. Can training generative AI models qualify the grounds of 'research' and claim exception under Section 52 and get immunity from copyright infringement. Several countries like European Union, Japan, Singapore, UK have adopted text and data mining (TDM) exception that legitimizes copying of large volumes of data for the purposes of AI training and machine learning. But legitimized usage of datasets severely undermines the attribution rights of the human creators.

Misuse of Artificial Intelligence leads to deep fakes, fake videos, non-consensual intimate images, misleading political content, child sexual abuse material, revenge porn etc. The foundation of criminal jurisprudence lies on proving the act beyond reasonable doubt. In such AI generated deep fakes, it is difficult to prove the history of such material including finding legitimate answers on who created it? Was the content modified by AI? Was the content altered after being generated by AI? etc. This is seen more of a technological vacuum more than a legal vacuum.

Global Emerging Trends in AI Regulation

The EU Artificial Intelligence Act (2024) is the world's first comprehensive legal framework for Artificial Intelligence where the AI systems are classified into four basic levels of risk and safeguards are imposed based on the level of risk that the systems impose on the society. Further the Act establishes European AI Office that supervises foundation models and coordinates enforcement, National market surveillance authorities, European Artificial Intelligence Board that provides harmonization & guidance.

Table 1:  Risk Categories Under the EU Artificial Intelligence Act and Illustrative Use-Case Scenarios

In United States, National Institute of Standards and Technology (NIST) – AI Risk Management Framework (AI RMF) was released in January 2023. This was a voluntary guidance aimed at organizations that are either developing AI systems or deploying AI systems in their products and services. The framework defined best AI practices, protecting AI systems from cyber threats, how to deploy privacy and data protection within the developmental stages of AI, specifics of ethical AI, data integrity, various risk AI specific risk assessment tools, algorithmic accountability and impact quantification of the AI systems.

Canada's Artificial Intelligence and Data Act, which is still in the process of legislative debate, emphasizes curtailing AI related risks and discrimination while supporting the economic growth and global competitiveness in AI research. Unlike EU AI Act, Canadian legislators adopted a technology neutral approach and relies on transparency and data disclosures by the data fiduciaries.

Way Forward for India: A Techno-Legal Approach     

While we recognize the growth of technology is multifold and, in many cases, the legal systems lag behind to enforce safe use and protection of individual privacy. The generative AI has altered the extent of digital risk in ways that existing Indian statutes were not designed to anticipate. The Information Technology Act, 2000, and the Digital Personal Data Protection Act, 2023, continue to provide broad foundational governance. These statutes are anchored in a regulatory philosophy of static content, predictable data flows and human-initiated actions. Hence there is a growing need to evolve the framework towards evolving dynamic technological environment. 

A techno-legal approach involves using technology to enforce legal requirements automatically. The legal framework and governing principles are embedded while designing of the digital systems and hence technology develops products and services that are legally compliant. This ensures automatic compliance, reduces human error, helps in preventing harm, not just punish it later by allowing the harm to occur. The techno legal approach provides ex-ante safeguards like privacy by design, safety by design, such that compliance is inbuilt within the design of the system rather than left to discretionary human intervention. This approach reduces the human dependency and allows the regulators to identify, trace and audit the safeguards through the AI lifecycle.

Currently, In India Data Empowerment and Protection Architecture (DEPA) model that was inspired by and built on top of the principles learned from UPI and Aadhaar is being discussed towards AI training. This will allow AI models to adopt consent and privacy requirements automatically, allows the algorithms to log and share on how data was used, checking if the data used for training is copyrighted material or not. In conclusion, India must bridge its legal and technological gaps to ensure that the growth of AI dominance as a tool for empowerment rather than a source of harm.

• By Sriharini Chellappan