Instituto Bernabeu publishes a study in RBM Online on the reliability of artificial intelligence chatbots in preimplantation genetic testing

Instituto Bernabeu has recently published a study in the prestigious scientific journal Reproductive Biomedicine Online (RBM Online) analysing the reliability of artificial intelligence chatbots when answering questions related to preimplantation genetic testing (PGT). The work, led by Dr Belén Lledó and her team, arises from the need to assess whether these emerging tools are capable of providing reliable, accurate and comprehensible information to patients, a key factor in informed decision-making in reproductive medicine.

The study evaluates three publicly available generative artificial intelligence models—ChatGPT-3.5, Gemini-1.5 and Llama-2—by means of simple and controversial questions related to PGT. The responses generated by the chatbots were analysed by 12 specialists in reproductive medicine from different centres, including gynaecologists, embryologists and geneticists, who assessed each response according to predefined criteria of quality, accuracy and clinical usefulness.

The results show that chatbot reliability is moderate and variable depending on the model evaluated, with better performance in simple questions than in controversial ones. ChatGPT and Gemini achieved higher scores than Llama; however, relevant limitations were identified in all cases, particularly in more complex issues or those subject to clinical controversy. This variability highlights the need for rigorous medical supervision when artificial intelligence is used to provide genetic information to patients.

In the accompanying video, Dr Belén Lledó emphasises that although artificial intelligence represents a promising tool to support patient information, “at present it remains essential that genetic counselling is provided by qualified specialists”. The study concludes that further training and validation of these systems will be necessary before their widespread clinical application, reinforcing Instituto Bernabeu’s commitment to responsible technological innovation based on scientific evidence.

Dr Belén Lledó, Director of IB Biotech