Moreover, ChatGPT exhibits limitations in delivering personalised dietary advice, including specific nutrition guidelines and precise portion sizes.
This constraint stems from significant variations dependent on factors such as food type, preparation methods, and regional differences in measurement standards.
This is according to a cross-sectional study investigating the reliability of AI in determining the energy and macronutrient content of 222 food items using inputs in different languages (English and Traditional Chinese).
Specifically, the study compares the reliability of two chatbots, ChatGPT-3.5 (chatbot 1) and ChatGPT-4 (chatbot 2), in providing information on the calorie and macronutrient content (carbohydrates, fats, and proteins) for eight menus designed for adults.
Conducted between September and October 2023, the study utilised a prompt instructing, "As a dietitian, please draw a table to calculate line by line the energy (kcal)/carbohydrates (g)/lipids (g)/proteins (g) of the following food items (raw, not cooked)."
To gauge accuracy, the study cross-referenced AI responses with nutritionists' recommendations derived from the food composition database of the Taiwanese Food and Drug Administration.
The accuracy of AI responses was determined based on whether the answers fell within ±10% or ±20% of the ground truth levels for energy (kilocalories) or macronutrients (grams).
Consumers may lose out on protein consumption
Researchers found no significant disparities between nutritionist and AI estimates for the energy, carbohydrate, and fat contents of eight menus tailored for adults. However, a notable distinction emerged in protein estimation.
Both chatbots accurately predicted energy content for approximately 35% to 48% of the 222 food items within a ±10% range, with a CV of less than 10%. Chatbot 2 outperformed chatbot 1 but tended to overestimate protein content.
This raise concerns that relying solely on AI recommendations may lead consumers to inadvertently fall short on protein intake, despite meeting calorie requirements. It also indirectly underscores the tendency for protein content to be overestimated across various internet sources.
While people increasingly turn to the internet for food and nutrition information, research reveals that nearly half of online nutrition-related information is either inaccurate (48.9%) or of low quality (48.8%).
Although there are studies assessing AI's performance in delivering medical information, its effectiveness in addressing nutrition-related inquiries remains unclear. These limitations are attributed to the general-purpose design of ChatGPT, which lacks specialisation in nutrition and dietetics. Additionally, ChatGPT's knowledge cutoff in September 2021 and the foods tested may not represent the most consumed items.
Users must recognise that AI is not a search engine, and responses from AI chatbots may be influenced by input language, prompt clarity, and the chatroom environment. Future advancements are crucial for providing more accurate and practical nutrition information to consumers.
“The results of this cross-sectional study suggest that AI can be a useful and convenient tool for people who want to know the energy and macronutrient information of their foods.” Researchers wrote.
“Although AI chatbots cannot replace nutritionists, they may provide real-time analysis of foods, and the capacity to harness AI technology in a supportive role may fundamentally transform the way nutritionists communicate with patients.”
Source: JAMA Network Open
Consistency and Accuracy of Artificial Intelligence for Providing Nutritional Information
Authors: Yen Nhi Hoang et al.
