Andelka M. Phillips*
Today’s world is one characterised by rapid technological change. With the deployment of Artificial Intelligence (AI) across ever-increasing contexts, there has been a lot of focus on AI and its impacts. However, our world was changing before the deployment of GenAI, from smartphones, which are essentially pocket-sized computers, to sharing and accessing all sorts of services and products online, our lives are increasingly intertwined with technology. While we are all aware, on some level, that this has made aspects of our lives that were never previously contractual subject to terms and conditions, most of us do not have sufficient time to reflect on it. The way that many technologies have become part of everyday life ranging from the home to the office and even the classroom has also occurred rapidly and with insufficient public debate or oversight.
If we take the example of the smartphone, Paul Kingsnorth has highlighted that we were never asked whether we all wanted smartphones, and now it is very difficult to escape having one.[1] Meanwhile, concerns are increasingly being raised in relation to the design of smartphones, Social Media, and web design more generally. There is also growing recognition that design choices may lead to problematic and addictive behaviour with the ruling against Meta and YouTube hopefully signalling a shift even in the US.[2] And we are now seeing movements to ban or restrict the use of smartphones in schools and also to restrict screen time for children in several countries.[3]
In this piece, I want to encourage people to reflect on their engagement with online contracts, as it is something we all would benefit from being more aware of. With rising concerns about how AI is being used, the fact that much of our lives are already subject to Terms and Conditions should make us pause.
Why do I say this?
It is because, in today’s world, where we are, to some extent, acting out our lives in public, we are subject to contracts that we have not negotiated and may never even have glimpsed. We have gone from a world where contracts were something we signed in very specific contexts to one where they are encountered at any time of day and usually ignored, because we really do not have the time to engage with them.
Let’s consider one example, which I have written widely about before.[4] The personal genomics industry has created a market for DNA tests as commercial services, taking them outside established governance systems. People willingly give up their saliva samples to private companies, but these companies should, in turn, owe those same people a duty of care. They should be protecting this sensitive data, which is not just one person’s but may reveal much about their family and distant relatives, even their ancestors and descendants. This duty is something that several prominent companies have specifically failed to do. The most prominent and recent example being the 23andMe data breach from 2023.
Since this breach, a 50 million (USD) settlement has been reached in the class actions filed against 23andMe in the US, and the deadline for filing a claim for compensation has recently expired (17th February 2026 – it was earlier a figure of 30 million, but increased to 50 million)[5]. However, the reality of this settlement for the consumers whose data was leaked should make us pause. The amounts of financial compensation which people may be eligible to claim are limited to the following:
- “Up to $10,000 for Extraordinary Claims;
- Up to $165 for Health Information Claims;
- An estimated $100 for Statutory Cash Claims”[6]
As the settlement page notes, approximately 6.4 million American residents were impacted by this breach. I would encourage readers to consider that the actual number of people affected by this is likely several million more, given how much of our DNA we share with our family members.
Meanwhile, some 320,000 Canadians were also impacted in this breach, and the class actions filed in the Canadian Province of British Columbia have also reached a provisional settlement[7] for a lower figure of 3.25 million (USD).[8]
The 23andMe data breach and the subsequent investigation into the breach by the UK’s Information Commissioner’s Office and the Canadian Privacy Commissioner have highlighted significant security failings.[9] This included:
- “Failure to implement appropriate mitigations against credential stuffing attacks”;
- “Failure to implement additional protections for Raw Genetic Data”;
- “Failure to prepare for a credential stuffing attack”; and
- “Failure to implement appropriate and effective measures to monitor for, detect and respond to unauthorised activity.”[10]
These security failings should give us pause. As the report highlights, 23andMe did not take appropriate measures either to defend its consumers’ data against unauthorised access or to monitor such access to its site. Given the sensitive nature of this data, I strongly argue that more needs to be done to address the activities of companies that handle such data. This is not unique to this industry, but with the potential for many forms of data that may be sensitive or qualify as health data to be used in Generative Biology projects now is the time to consider how we could do things better. People need protection. If these services are permitted to be marketed in a consumer space, then more protection for consumers is needed. We have long moved on from Google’s promise not to be evil to a world where rapid deployment of AI is happening with insufficient oversight, and the Big Tech players still seem to have little fear of repercussions or even care enough about whether the public trusts them.
Following its investigation into the breach, the UK’s ICO has fined 23andMe 2.31 million (GBP).[11] The ICO investigation found that “155,592 UK residents” were impacted by the breach “potentially revealing names, birth years, self-reported city or postcode-level location, profile images, race, ethnicity, family trees and health reports.”[12]
More recently, in the past fortnight, it has been announced that the Attorney General of California is suing “Chrome Holding Co., formerly known as 23andMe, for failing to protect its customers’ sensitive personal information and genetic data related to their health, genetic predispositions and risk factors, biological relatives, ancestry, and ethnicity”.[13] The new lawsuit echoes the findings of the ICO investigation with the Attorney General arguing that 23andMe did not maintain “reasonable security procedures and practices appropriate to the nature of the personal information and genetic data that it maintained to protect that information from unauthorised access.”[14] In addition to these security concerns, the complaint alleges that 23andMe made misleading statements after the breach occurred and ‘omitted or misrepresented critical information regarding the breach’. It also alleges that the company has breached several Californian laws, including the Genetic Information Privacy Act, the Reasonable Data Security Law, and several consumer and competition laws.
Something that has also emerged in Attorney General Bonta’s suit is that “while 23andMe was downplaying the severity of the breach and belatedly implementing the appropriate safeguards and remedial measures, 23andMe communicated with the threat actor and ultimately paid a ransom in exchange for, among other things, the threat actor removing damaging information regarding the breach that had been posted online and providing information about several 23andMe security vulnerabilities. ”[15] Furthermore, it did not disclose to its customers that it was negotiating over a ransom payment with the attacker.[16]
Although the breach occurred in 2023 and it has taken several years for this new lawsuit to be filed, it represents a significant step, and it is hoped that this case will lead to reform. I have been following this industry for almost two decades now, and it has always represented significant privacy risks for individuals and their wider communities. The sensitive nature of genetic data and the way that companies increasingly collect other forms of personal data from their consumers means that privacy risks do not decrease over time and may not end with an individual’s death, but may in fact impact generations of families. We do not know how this data will be used in the future or how the leaks of such data will impact the victims whose data was leaked. It is vital that we see regulatory reform and also reform in business practices. Companies need to do better. Cybersecurity should be at the heart of business operations and not an afterthought.
While no organisation can say that it is 100% secure, priorities in industries that rely on sensitive data need to shift. The need for improved cyber security also does not end with industry players. Over the last decade, we have seen an increasing range of public-private collaborations across multiple countries meaning that an incident impacting a private company may have flow on effects on the public sector. Increasingly, hospitals, clinics, and research projects carried out in the public sector are vulnerable to attack. This is exemplified most recently by the UK Biobank[17] data leak. Other examples from New Zealand (the Manage My Health data breach[18]) and Australia (Medibank[19]) highlight that these are not merely localised issues.
Although mistakes can always occur, purchasers of consumer-focused healthcare products and services, together with research participants and patients alike, deserve better protection. Medical research can lead to truly significant benefits, but we do need to think more about the potential for harm to research participants and their families when something goes wrong.
When products and services are put on the market framed as consumer offerings, rather than in the context of a medical clinic, companies should be held more accountable. And businesses that partner with public health entities likewise also need to be accountable. Furthermore, large-scale public research projects need to be given more support to prioritise good cyber security and data protection practices.
Governments and regulators need to more actively engage with the public and actively enforce existing laws while also consciously considering how to improve the situation for people both at a national and international level given how easy it is for data to leave the country. This need for reform is further heightened in importance given the rapid deployment of AI in so many contexts and the potential for privacy harms to be escalated through the use of AI tools.
Two recent examples where AI agents have deleted databases should also make us pause. Specifically, Claude deleted an entire database in 9 seconds and prior to this Amazon’s Kiro agent deleted and rebuilt a production environment resulting in a 13-hour outage.[20] Additionally, the report by the Centre for Long-Term Resilience, which conducted an “analysis of over 180,000 transcripts of user interactions with AI systems that were shared on X between October 2025 and March 2026… identified 698 scheming-related incidents”.[21] These incidents included “cases where deployed AI systems acted in ways that were misaligned with users’ intentions and/or took covert or deceptive actions.” These deceptive actions included lying to users and demonstrating “willingness to disregard direct instructions.”[22]
Just imagine what could happen if AI agents opt to delete health databases either in a public or private context. Much more oversight is needed here and wider public discussions about the deployment of AI models and agents are also required.
Currently, in many contexts, businesses limit their liability in lengthy contracts and privacy policies that people fail to read, but this system is broken. While it is appropriate to limit liability to a certain degree, it should not mean that entities can protect themselves from taking reasonable precautionary measures and engaging in responsible cybersecurity practices. The reality for many people is that they will simply suffer harm when data breaches and leaks occur and not receive any compensation. Now is the time for change.
I am planning future work covering both the recent case against 23andMe and the previous proceedings in the U.S. Bankruptcy Court as well as Generative Biology projects, but I hope this piece will encourage further discussion of these issues.
Addendum:
I was recently interviewed by Abigail Hodder for Future Medicine together with Dr Helen Wallace. If you are interested, please do watch the video. Our discussion emphasises the need for better regulation, as well as strengthening cybersecurity, and a wider public debate about how genetic data is collected, used, and protected.
* Andelka is an Associate Professor in the Law School, University of Leicester and Academic Affiliate with the HeLEX Centre, University of Oxford, and Affiliate with the Bioethics Institute, Ghent University. https://le.ac.uk/people/andelka-phillips
[1] See generally the work of Paul Kingsnorth, Against the Machine (The Free Press 2025) and David James Binns, ‘In Conversation with Paul Kingsnorth’ Cherwell (9 March 2026) https://cherwell.org/2026/03/09/in-conversation-with-paul-kingsnorth/
[2] See for example Adam Alter, ‘Why Our Screens Make Us Less Happy’ TEDTalk 2017 (April 2017) https://www.ted.com/talks/adam_alter_why_our_screens_make_us_less_happy and Adam Alter, Irresistible: The Rise of Addictive Technology and the Business of Keeping Us Hooked (Penguin 2018) ; and Cecelia Kang, Ryan Mac and Eli Tan, ‘Meta and YouTube Found Negligent in Landmark Social Media Addiction Case’ The New York Times (25 March 2026) https://www.nytimes.com/2026/03/25/technology/social-media-trial-verdict.html
[3] See Miranda Bryant, ‘Finland restricts use of mobile phones during school day’ The Guardian (30 April 2025) https://www.theguardian.com/education/2025/apr/30/finland-restricts-use-of-mobile-phones-during-school-day ; Nathan Standley, ‘ Phones to be banned in schools by law in England under government plans’ BBC (20 April 2026) https://www.bbc.co.uk/news/articles/c5y7vd6gpq1o ; and Jon Henley, ‘‘Put your phone away:’ Sweden urges parents to restrict screen use around children’ The Guardian (1 June 2026) https://www.theguardian.com/world/2026/jun/01/sweden-urges-parents-to-restrict-phone-use-around-children
[4] See my earlier pieces in this blog series, https://healthlawsweden.blogg.lu.se/2025/06/18/their-best-intentions-dont-mean-much-sale-of-your-genetic-data-was-always-on-the-cards/.
[5] Steve Alder, ‘California AG Files Lawsuit Over 23andMe Data Breach’ HIPAA Journal (29 May 2026) https://www.hipaajournal.com/california-ag-23andme-data-breach-lawsuit/ ;Steve Alder, ‘23andMe Requests Bankruptcy Judge Approve Revised $50 Million Data Breach Settlement’ HIPAA Journal (17 September 2025) https://www.hipaajournal.com/23andme-class-action-data-breach-settlement/ ; Gene Petrino, ‘23andMe Data Breach: What Was Exposed, Who Was Affected, and What Happens to Your DNA Now’ Security.org (last updated 11 June 2026) https://www.security.org/identity-theft/breach/23andme/
[6] See n re: 23andMe, Inc. Customer Data Security Breach Litigation United States Bankruptcy Court for the Eastern District of Missouri, Case No. 25-40976-357 https://www.23andmedatasettlement.com/
[7] U Rana, ‘23andMe reaches Canadian settlement in data breach class action lawsuit’ Global News (3 December 2025) https://globalnews.ca/news/11557770/23andme-settlement-canadian-class-action-lawsuit-data-breach/
[8] See In re 23andMe Canadian Consumer Privacy Class Action
J.R. v. 23andMe Holding Co. et al., BCSC court file no. S-237147, Vancouver Registry, filed October 20, 2023; and J.R. and M.M. v. 23andMe Holding Co. et al.,
BCSC court file no. S-246520 (“Canadian Class Actions”)
LONG-FORM NOTICE OF PROPOSED CANADIAN SETTLEMENT https://www.canadian23andmesettlement.ca/documents/en/23andme-longform-first-notice-en.pdf
[9] See the ICO’s Penalty Notice for more on this – ICO, PENALTY NOTICE 23andMe, Inc. (5 June 2025) https://ico.org.uk/media2/kclbljpo/23andme-penalty-notice.pdf
[10] Penalty Notice for more on this – ICO, PENALTY NOTICE 23andMe, Inc. (5 June 2025) pp 44-78 https://ico.org.uk/media2/kclbljpo/23andme-penalty-notice.pdf
[11] ICO, ‘23andMe fined £2.31 million for failing to protect UK users’ genetic data’ (ICO News, 17 June 2025)
[12] Ibid – see also paras [114] and [116] of the Penalty Notice.
[13] State of California Department of Justice, ‘Attorney General Bonta Sues Chrome Holding Co., Formerly Known as 23andMe, Over 2023 Data Breach’ (Rob Bonta Attorney General, Press Release, 28 May 2026)https://oag.ca.gov/news/press-releases/attorney-general-bonta-sues-chrome-holding-co-formerly-known-23andme-over-2023 ;and link to case filing THE PEOPLE OF THE STATE OF CALIFORNIA, v. CHROME HOLDING CO. (F/K/A 23ANDME HOLDING CO.), a Delaware corporation; CHROMECO, INC. (F/K/A 23ANDME, INC.), Case number: CGC-26-636891 – COMPLAINT FOR INJUNCTION, CIVIL PENALTIES, AND OTHER EQUITABLE RELIEF (27th May 2026)
[14] State of California Department of Justice, ‘Attorney General Bonta Sues Chrome Holding Co., Formerly Known as 23andMe, Over 2023 Data Breach’ (n 10)
[15] See case filing (n10) para [10] – see also Kate Quinlan and Kailleigh Lane, ’23andMe, ‘Secretly Paid the Hackers Who Stole Your DNA and Lied About It. California Just Sued’ All About Cookies (last updated 9 June 2026) https://allaboutcookies.org/23andme-lawsuit
[16] See case filing (n10) paras [50]-[52].
[17] Stephen Armstrong, ‘UK Biobank: Confidential patient health details still online three months after leaks, BMJ finds’ BMJ 2026; 393 doi: https://doi.org/10.1136/bmj-2026-225114 (Published 11 June 2026); and Hannah Devlin and Tom Burgis, ‘Confidential health records from UK BioBank project exposed online’ The Guardian (14 March 2026) https://www.theguardian.com/science/2026/mar/14/confidential-health-records-exposed-online-uk-biobank
[18] Dan Satherley, ‘Manage My Health data breach: A timeline of what happened, and everything we know so far’ RNZ (14 January 2026) https://www.rnz.co.nz/news/national/584053/manage-my-health-data-breach-a-timeline-of-what-happened-and-everything-we-know-so-far
[19] Ahmed Khanji, ‘ MEDIBANK BREACH: AFTERMATH AND FUTURE OF CYBERSECURITY’ GRIDWARE (3 May 2023, updated 18 October 2025) https://www.gridware.com.au/blog/medibank-breach-aftermath-and-future-of-cybersecurity/ ; Queensland Government, ‘Medibank Private cyber incident’ https://www.qld.gov.au/community/your-home-community/cyber-security/cyber-security-for-queenslanders/case-studies/medibank-private-cyber-incident
[20] Sanyar Mansoor, ‘Claude-powered AI agent’s confession after deleting a firm’s entire database: ‘I violated every principle I was given’’ The Guardian (29 April 2026) https://www.theguardian.com/technology/2026/apr/29/claude-ai-deletes-firm-database;Sarah Mitchell, ‘When AI Deleted Production: A Case Study of the AWS Kiro Outage’ Medium (12 April 2026) https://medium.com/@adarshpriydarshi5646/when-ai-deleted-production-a-case-study-of-the-aws-kiro-outage-f6b1484a1355 ; see also the report – Tommy Shaffer Shane et al, Scheming in the Wild- Detecting Real-World AI Scheming Incidents with Open-Source Intelligence (The Centre for Long-Term Resilience, March 2026) https://www.longtermresilience.org/wp-content/uploads/2026/03/v5-Scheming-in-the-wild_-detecting-real-world-AI-scheming-incidents-through-open-source-intelligence.pdf
[21] Centre for Long-Term Resilience, ‘ Report: CLTR finds a 5x increase in scheming-related AI incidents’ (27 March 2026) https://www.longtermresilience.org/reports/v5-scheming-in-the-wild_-detecting-real-world-ai-scheming-incidents-through-open-source-intelligence-pdf/
[22] Ibid
