Google made significant strides in artificial intelligence at the I/O 2024 event. Here’s a glimpse into Gemini 1.5 Flash, Project Astra, Imagen 3, and more…
Gemini Gets Faster and Smarter!
Google DeepMind CEO Demis Hassabis announced updates to the Gemini model family. Initially launched in December, the first local multimodal model offered in three sizes (Ultra, Pro, Nano) — Gemini 1.0 — was soon followed by the enhanced performance and a 1 million token extended context window of the 1.5 Pro version.
Developers and enterprise users found the long context window, multimodal reasoning capabilities, and overall performance of 1.5 Pro highly beneficial. In response to user feedback indicating that some applications require lower latency and service cost, Google introduced a new member to the Gemini family: 1.5 Flash.
Gemini 1.5 Flash
Optimized for speed and efficiency, this lightweight model is ideal and cost-effective for high-volume, high-frequency tasks. Providing an extended context window of 1 million tokens, 1.5 Flash excels in tasks like summarization, chat applications, image and video captioning, data extraction from long documents and tables, among others. Trained using the “distillation” method by the larger 1.5 Pro model, 1.5 Flash transfers fundamental knowledge and skills to a smaller and more efficient model.
Gemini 1.5 Pro
Google also significantly improved the best model for overall performance: 1.5 Pro. The context window has been expanded to 2 million tokens. Data and algorithmic enhancements have improved code generation, logical reasoning and planning, multi-turn conversation, and audio and visual understanding features.
1.5 Pro can now handle increasingly complex and nuanced instructions, including product-level behavioral determinants such as role, format, and style. Control over model responses for specific use cases, such as creating the personality and response style of a chat application or automating workflows through multiple function calls, has been enhanced. Users can now direct model behavior by adjusting System instructions.
Gemini Nano
Going beyond just text inputs, Gemini Nano, can now process images as well. Applications using Gemini Nano with Multimodality will be able to understand the world like humans do, not only with text input but also with voice and speech.
The Future of AI Assistants: Project Astra
As part of its mission to develop AI responsibly to benefit humanity, Google DeepMind announced Project Astra, aimed at developing universal AI agents that can assist in everyday life. Astra aims to develop AI agents that can understand and act on context like humans, helping them understand and respond to the complex world.
These agents will serve as proactive, accessible, and personalized assistants. Users will be able to converse with these agents naturally and without delay. Designed to process video and speech inputs and remember them, Astra’s features will be integrated into Google products like Gemini applications later this year.
New Productive Media Models and Tools
Google also introduced new productive media models and tools for creative work:
Veo
Google’s most capable video creation model to date, Veo, can create high-quality 1080p videos lasting more than a minute. Supporting various cinematic and visual styles, Veo understands natural language and visual semantics to create videos that reflect the user’s creative vision. The model also provides unprecedented creative control by understanding cinematic terms like “timelapse” or “aerial shot of a landscape.”
It creates consistent and coherent shots; people, animals, and objects move realistically throughout the shots. Google invites a range of filmmakers and content creators to try the model to explore how Veo can best support the storyteller’s creative process.
Imagen 3
Google’s highest-quality text-to-image model, Imagen 3, can produce incredibly detailed and photorealistic images. By understanding natural language, Imagen 3 can better interpret user commands, including small details from long commands and generating text within an image.
Music AI Sandbox
A set of tools supporting musicians in creating new instrumental pieces from scratch, transforming sound, and supporting creative work, Music AI Sandbox aims to open up a new playground for creativity.
AI Developments
Google not only focuses on advancing technology but also on addressing the challenges posed by productive technologies and helping people work responsibly with AI-generated content. These measures include collaborating with the creative community and other stakeholders, gathering insights to develop and distribute technologies safely and responsibly, listening to feedback, and giving content creators a voice. Google believes that AI technologies should be used to benefit humanity and is working to ensure that these technologies are developed ethically, responsibly, and fairly.
