AI Struggles with Telling Time: A New Study Highlights Gaps in Multimodal Language Models’ Abilities
In today’s world, artificial intelligence (AI) is capable of remarkable feats, from generating photorealistic images to writing novels and even predicting protein structures. However, recent research has revealed a surprising weakness in AI: its inability to accurately read clocks and calendars. Despite its impressive capabilities, AI is failing at a very basic task—telling time.
The Study: Testing AI’s Time-Reading Skills
Researchers from the University of Edinburgh conducted a study to examine the performance of seven well-known multimodal large language models (MLLMs), which are AI systems capable of interpreting and generating various types of media. Their goal was to test how well these models could answer time-related questions based on images of clocks and calendars. The study, set to be published in April and currently available on the preprint server arXiv, shows that even advanced AI struggles with these simple tasks.
“The ability to interpret and reason about time from visual inputs is critical for many real-world applications—ranging from event scheduling to autonomous systems,” said the researchers in their paper. “Despite advances in multimodal large language models (MLLMs), most work has focused on object detection, image captioning, or scene understanding, leaving temporal inference underexplored.”
The AI Models Tested
The study tested several high-profile multimodal language models, including:
- OpenAI’s GPT-4o and GPT-o1
- Google DeepMind’s Gemini 2.0
- Anthropic’s Claude 3.5 Sonnet
- Meta’s Llama 3.2-11B-Vision-Instruct
- Alibaba’s Qwen2-VL7B-Instruct
- ModelBest’s MiniCPM-V-2.6
These models were shown a variety of images, including analog clocks (with different styles, Roman numerals, missing seconds hands, and other variations) and calendar images spanning 10 years. The researchers then asked the AI models to answer questions related to time, such as determining the time shown on a clock or identifying specific dates on a calendar.
AI’s Struggles with Clock Reading and Calendar Comprehension
The research revealed that the AI systems performed poorly across the board. When asked to read the time on analog clocks, they were correct less than 25% of the time. The models struggled with clocks featuring Roman numerals, non-standard hand positions, and clocks missing a seconds hand. These results suggest that the AI’s difficulties might stem from its inability to detect clock hands or accurately interpret the angles on a clock face.
The models were also tested with calendar-related questions. For example, they were asked basic queries like “What day of the week is New Year’s Day?” as well as more complex questions such as “What is the 153rd day of the year?” While some models, such as GPT-o1, scored better than others, they still made mistakes about 20% of the time.
Google’s Gemini-2.0 performed the best in the clock task, while GPT-o1 achieved an 80% accuracy rate on the calendar task—still far from perfect. Even the most successful model in the study struggled with a significant portion of the tasks.
Implications for AI’s Real-World Applications
“Most people can tell the time and use calendars from an early age. Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people,” said Rohit Saxena, a co-author of the study and PhD student at the University of Edinburgh’s School of Informatics. “These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies.”
This research sheds light on the limitations of AI in real-world applications that rely heavily on accurate time-keeping. Although AI may be able to assist with complex tasks like homework or even medical predictions, it still struggles with the most fundamental aspects of human daily life.
Conclusion: AI’s Limitations and Future Challenges
While AI is advancing rapidly and showing impressive capabilities in various domains, this study serves as a reminder of the challenges that remain. AI’s struggle with something as basic as reading clocks and calendars highlights the gaps in its understanding of time—a fundamental aspect of human life. For AI to be more successfully integrated into time-sensitive fields such as scheduling, automation, and assistive technologies, these gaps must be addressed.
So, while AI can undoubtedly help you with homework, you might want to double-check the time yourself.
For more information, visit the arXiv preprint server.
References:
- Saxena, R., et al. (2025). “Multimodal AI Models Struggle with Time Interpretation,” University of Edinburgh. arXiv.
- ChatGPT, OpenAI’s language model.
0 Comments