Generative AI

“Some Harm Considerations of Large Language Models (LLMs)” by Rebecca Sweetman is licensed under CC BY-NC-SA 4.0 International.

Some commentators have questioned the ethics of generative AI tools. Students, instructors and researchers may wish to be familiar with these objections when considering whether to engage with AI. Many of the major criticisms fall into the following themes:

Misinformation: 
Generative AI could increase the quantity of misinformation due to factually incorrect output and use by bad actors. GenAI is known to produce “hallucinations” – confident-sounding yet fictional sources of information. Numerous examples exist in government, news outlets, law, academia, and medicine of documents produced that contain fake sources that appear to support the overall position or argument. Building research on non-existent sources can lead to drawing incorrect conclusions and contributing to the spread of misinformation, while undermining the credibility of the author(s) or organization that produced the document. Carefully interrogating all sources provided in a GenAI output, and tracing any citation to its origin will help to ensure that you are using accurate, authoritative sources and weeding any hallucinations in your research.

Bias: 
The training data used by LLMs is often scraped from the web and is not curated for quality or neutrality. Due to the preponderance of English-language content on the Internet – the majority of which is produced in the US – LLMs are mainly trained on English text, meaning that GenAI outputs are likely to reproduce Western societal and cultural biases. To counter this, some researchers are advocating for the use of culturally diverse datasets to reduce bias. Others suggest that GenAI tools are more effective when there is greater “cultural alignment” between users and AI models, highlighting the need for cross-cultural AI literacy. When using GenAI tools, it is essential to remember that technology is not neutral, and to think critically about potential biases that are surfaced by LLMs.

Critical thinking: 
Research suggests that a reliance on GenAI in academic contexts could de-emphasize traditional academic skills and competencies and ultimately lead to a decrease in critical thinking skills. One study concluded that while use of GenAI can improve efficiency, it can have the effect of shifting critical thinking from deep engagement and synthesis to verifying sources. Another notes that GenAI use can improve fluency and flexibility, but also encourages passive acceptance of information seen as authoritative from the LLM, while potentially reducing self-confidence in critical judgment and creativity. There is potential concern for cognitive offloading, which refers to using external aids and tools to achieve certain cognitive tasks. GenAI use can make tasks faster and easier, but can also remove the opportunity for creativity, problem solving, and active recall, all of which are essential components of deep learning. A 2025 study by Microsoft found that greater confidence in GenAI was associated with less critical thinking (inverse correlation), while greater self-confidence is associated with higher levels of critical thinking (direct correlation). Studies suggest that the outcome depends on how you use the tool – it can be a crutch, or a tool for growth.

Copyright infringement: 
There are significant concerns over the use of copyrighted intellectual, creative, and artistic works in the training of LLMs. GenAI firms train their models on vast amounts of information scraped from the internet. In the process, they ingest the copyrighted works of scholars, authors, composers, artists, filmmakers, etc. without compensation or consent. GenAI companies claim that this falls under ‘fair dealing’ (Canada) or ‘fair use’ (US) exemptions in copyright law that state that uses for research, private study, education, parody, or satire do not infringe on copyright. It should be noted that these companies and their users are able to profit off outputs produced using GenAI, depriving creators of recognition, royalties and other income.

User data: 
GenAI tools often require users to input data, which can include sensitive or personal information, which creates real ethical concerns over student privacy in requiring the use of these tools. Students might unintentionally share information that could be used for purposes that the user might not anticipate or consent to. AI systems are data-hungry and are generally not transparent about data collection practices or the uses to which scraped data is put. As LLMs require large non-synthetic datasets (i.e. data generated by human activity) for training their models, GenAI companies may have a strong incentive to use high-quality, student-generated datasets, potentially violating users’ privacy in pursuit of profit. For these reasons, it is recommended that students only use approved AI tools for use in their work, and engage with GenAI critically. Best practices include data minimization (provide only as much information as absolutely necessary), anonymization, and avoidance of personally identifiable data.

Business practices: 
Research by investigative journalists has shown that many GenAI companies maintain harmful business practices. Tasks such as data annotation and cleaning datasets (i.e. labelling text or images as harmful, offensive, inappropriate or illegal) are outsourced to low-paid workers, often in developing countries. Constant exposure to this content creates real emotional and psychological harm on the workers who are forced to engage with this content; in addition, the non-unionized work culture means there is very little support provided to workers who wish to protest such a hostile work environment, and little to no access to counselling.

Environmental impact: 
Large language models, and the data centres used to support them, are very resource intensive. They require huge amounts of compute (i.e. computational processing power) and consume massive amounts of electricity (to support processing) and fresh water (for cooling servers). These data centres are often built in vulnerable communities and/or Indigenous territories, and draw essential resources away from those who live nearby. In addition to the resource use and huge greenhouse gas footprint, research has shown that hyperscale data centres create “heat islands” which can raise the temperature of the surrounding area by up to 8-9°C. Some research suggests that GenAI uses 10-20 times the amount of energy compared to the same query using a search engine, which has a corresponding impact on greenhouse gas emissions.

Further reading:

Anshu, Bansal (2024, April 19). Is Your Data Safe With GenAI Practices? Here's How To Stay Secure. Forbes. https://www.forbes.com/councils/forbestechcouncil/2024/04/19/is-your-data-safe-with-genai-practices-heres-how-to-stay-secure/ 

Bender, E.M., Gebru, T., McMillan Major, A., Schmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conerence on Fairness, Accountability, and Transparency. 610-623. https://dl.acm.org/doi/10.1145/3442188.3445922

Chan, G. (2025, Aug 05). GenAI helps students think fast. It's time universities teach them to think slow. The Straits Times. https://www.proquest.com/newspapers/genai-helps-students-think-fast-time-universities/docview/3236466523/se-2 

Cya, Michelle (2026, May 13). The New York Times Got Caught Using AI Hallucinations in Its Reporting. The Walrus. https://thewalrus.ca/the-new-york-times-got-caught-using-ai-hallucinations-in-its-reporting/

de Fine Licht, Karl. (2025), Rethinking the Ethics of GenAI in Higher Education: A Critique of Moral Arguments and Policy Implications. Journal of applied philosophy, 42(4), 1317-1337. https://doi.org/10.1111/japp.70026

Hao, Karen (2024, March 1). AI Is Taking Water From the Desert. The Atlantic. https://advance.lexis.com/api/document?collection=news&id=urn%3acontentItem%3a6BFV-D911-DYY9-J17X-00000-00&context=1519360&identityprofileid=JKNWRN27861 

Hao, Karen (2025). Empire of AI: Dreams and nightmares in Sam Altman's OpenAI (1st ed.). Penguin Press. https://go.exlibris.link/n8MccrWs 

Hao, Karen & Deepa Seetharaman (2023, July 25). Cleaning up ChatGPT takes heavy toll on human workers. The Australian (Online). https://www.proquest.com/newspapers/cleaning-up-chatgpt-takes-heavy-toll-on-human/docview/2841808399/se-2 

Hickerson, D., & Perkins, M. (2026). A peek behind the curtain: Using step-around prompt engineering to identify bias and misinformation in GenAI models. Ithaca: Retrieved from https://www.proquest.com/working-papers/peek-behind-curtain-using-step-around-prompt/docview/3304366067/se-2 

Jose, B., Cherian, J., Verghis, A. M., Varghise, S. M., S, M., & Joseph, S. (2025). The cognitive paradox of AI in education: between enhancement and erosion. Frontiers in psychology, 16. https://doi.org/10.3389/fpsyg.2025.1550621

Lee, Hao-Ping (Hank), Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks & Nicholas Wilson (2025, April 25). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. In CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26–May 01, 2025,Yokohama, Japan. https://doi.org/10.1145/3706598.3713778

Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C.D. and Ho, D.E. (2025, June), Hallucination-Free? Assessing the reliability of leading AI legal research tools. Journal of empirical legal studies, 22(2), 216-242. https://doi.org/10.1111/jels.12413

Miller, Katharine (2024, March 18). Privacy in an AI Era: How Do We Protect Our Personal Information? Stanford University Human-Centred Artificial Intelligence. https://hai.stanford.edu/news/privacy-ai-era-how-do-we-protect-our-personal-information 

Mineo, Liz (2025, November 13). Is AI dulling our minds? The Harvard Gazette. https://news.harvard.edu/gazette/story/2025/11/is-ai-dulling-our-minds/

Nasr, N. R., Tu, C.-H., Werner, J., Bauer, T., Yen, C.-J., & Sujo-Montes, L. (2025). Exploring the impact of generative AI ChatGPT on critical thinking in higher education: Passive AI-directed use or human–AI supported collaboration? Education Sciences, 15(9), Article 1198. https://www.mdpi.com/2227-7102/15/9/1198

Panettieri, Joe (2026, April 2). Generative AI Lawsuits Timeline: Legal Cases vs. OpenAI, Microsoft, Anthropic, Google, Nvidia, Perplexity, Salesforce, Apple and More. Sustainable Tech Partner. https://sustainabletechpartner.com/topics/ai/generative-ai-lawsuit-timeline/ 

Paddison, Laura (2026, March 30). Scientists have found an alarming environmental impact of vast data centers. CNN. https://www.cnn.com/2026/03/30/climate/data-centers-are-having-an-underrported 

Pelly, L. (2025). Mood machine: The rise of spotify and the costs of the perfect playlist (First One Signal Publishers/Atria Books hardcover;1; ed.). Atria, an imprint of Simon & Schuster, LLC. https://go.exlibris.link/ldmpffw0

Perrigo, B. (2023, Jan. 18). Exclusive: OpenAI used Kenyan workers on less than $2 per hour to make ChatGPT less toxic. Time. Retrieved from https://time.com/6247678/openai-chatgpt-kenya-workers/ [June 8, 2023].

Reguly, E. (2026). AI data centres’ voracious thirst for water makes their environmental footprint even bigger: The world’s fastest-growing industry is also turning into one of the biggest threats to natural resources. Toronto: The Globe and Mail. Retrieved from https://www.proquest.com/blogs-podcasts-websites/ai-data-centres-amp-8217-voracious-thirst-water/docview/3354184447/se-2 

Ridell, Jason A. (2025, July 13). The Hidden Cost of Information: Google Search vs ChatGPT. Medium. https://jasonariddell.medium.com/the-hidden-cost-of-information-google-search-vs-chatgpt-31184cdf1582 

Royce, Christine Anne & Valerie Bennett (2025, March 10). To Think or Not to Think: The Impact of AI on Critical-Thinking Skills. National Science Teaching Association. https://www.nsta.org/blog/think-or-not-think-impact-ai-critical-thinking-skills?srsltid=AfmBOopkUTvW2XdjGVsT8VGOUmad2xfIC8XjfWcayBZk2RbR1mzaNhq6 

Shwartz, Vered (2024, February 13). Artificial intelligence needs to be trained on culturally diverse datasets to avoid bias. The Conversation. https://theconversation.com/artificial-intelligence-needs-to-be-trained-on-culturally-diverse-datasets-to-avoid-bias-222811

Tan, C. N., Han, M., Chua, R. Y. J., & Cheng, C. (2026). AI models as cultural beings: Investigating AI cultural biases and the impact of cultural alignment on human–AI creative collaboration. PNAS Nexus, 5(4), 14. doi:https://doi.org/10.1093/pnasnexus/pgag084

Wang, Zituo, Jiayi Zhu, Zhuoyu Wang, Vivian Jiaying Zhai & Jianyu Wu (2026, June). Generative AI So White: Racial Biases in AI Imagery Across the United States and China. Science Communication, 48(3), 305-337. https://doi.org/10.1177/10755470261428542 

Zhang, Y., Tan, L., & O’Halloran, K. L. (2026). Bias in GenAI: Exploring cultural variations between large language models. Linguistics and Education, 92, 101515. https://doi.org/10.1016/j.linged.2026.101515

Zimmerman, Asha (2023, April 8). A Ghostwriter for the Masses: ChatGPT and the Future of Writing. Annals of surgical oncology, 30, 3170-3173. https://doi.org/10.1245/s10434-023-13436-0