Understand Google api.ai and build Artificial Intelligent Assistant

Getting things done and keeping in touch with friends and family has never been easier.

Well promoted by Google, what can be more beautiful than imitating our intelligence to better our lives? Good Work!

Artificial Intelligence is trending and the world of artificially intelligent assistants is growing — Siri, Cortana, Alexa, Ok Google, Facebook M, Bixby – are some available by known tech leaders. Voice-enabled applications are in a lot of action recently with voice-activated speaker devices likely to get more user acceptance. It is significant to explore this ecosystem early enough to help create the optimum voice experiences as the field matures.

In this post, we will be getting to know specifically about Google Home and API.AI, a platform to build conversational assistant powered by NLP and Machine learning.i

Google Home is voice-activated speaker device powered by the Google Assistant.
Ask it questions.
Tell it to do things.
And with support for multiple users, it can distinguish our voice from others in your home so we get a more personalized experience. However, it is fascinating to realize that it’s quite easy to build our own AI assistant too, customize it to our own needs, our own IoT connected devices, our own custom APIs. The sky’s the limit.

Google opened up the Google Assistant platform for developers in December and currently, the platform supports building out Conversation Actions for the Google Home device. It is widely expected that the same Actions will eventually be available across Google’s other devices and applications.

Screen Shot 2017-05-23 at 11.41.47 AM
Image credits: Google home

api.ai (formerly Speaktoit) is a developer of human–computer interaction technologies based on natural language conversations. It provides conversational user experience platform enabling brand-unique, natural language interactions for devices, applications, and services. Developers can use API.AI services for speech recognition, natural language processing (intent recognition and context awareness), and conversation management to quickly and easily differentiate their business, increase customer satisfaction and improve business processes.
It is acquired by Google in September 2016, it provides tools to developers building apps (“Actions”) for the Google Assistant virtual assistant.

How do they do it

Image credits: Google

To build chatbots or conversation assistant, one of the first things to consider is conversation workflow management.  It’s the layer in your bot stack that handles all your natural language processing needs. Whenever a user types or talks something to bot, you need a good conversation workflow management tool to help you deal with the messiness of human verbal communication.


api.ai provides us with such a platform which is easy to learn and comprehensive to develop conversation actions. It is a good example of the simplistic approach to solving complex man to machine communication problem using natural language processing in proximity to machine learning.

Some key concepts:

Agents, NLU (Natural Language Understanding) modules for applications. Their purpose is to transform natural user language into actionable data and can be designed to manage a conversation flow in a specific way. Agents are platform agnostic. You only have to design an agent once and then can integrate it with a variety of platforms using our SDKs and Integrations, or download files compatible with Alexa or Cortana apps.

Machine Learning, allows an agent to understand user inputs in natural language and convert them into structured data, extracting relevant parameters. In the API.AI terminology, the agent uses machine learning algorithms to match user requests to specific intents and uses entities to extract relevant data from them. The agent learns from the data you provide in it (annotated examples in intents and entries in entities) as well as from the language models developed by API.AI. Based on this data, it builds a model (algorithm) for making decisions on which intent should be triggered by a user input and what data needs to be extracted. The model is unique to the agent. The model adjusts dynamically according to the changes made in agent and in the API.AI platform. To make sure that the model is improving, the agent needs to constantly be trained on real conversation logs.

Intent represents a mapping between what a user says and what action should be taken by software.

Entity represents concepts and serves as a powerful tool for extracting parameter values from natural language inputs. The entities that are used in a particular agent will depend on the parameter values that are expected to be returned as a result of agent functioning.

More concepts …

API.AI relation to other components & process flow

Image credit

Getting to know api.ai and getting started …

Easy to Learn:

“Ok Google. Let’s get started with API.AI”
These videos overview and tutorials help to get acquainted with this platform. It is useful to get the understanding of the platform and its intent not just in development perspective but how it all impact daily life and can be used to drive the development in a good direction.

It will be helpful to think about the conversational flow that we are expecting to happen, maybe get pen and paper,  draw it, then include the components accordingly to build that flow using API.AI.

Conversation workflow with Google home, Google assistant, and api.ai

Image credits: Google

Easy to Develop:

“Ok Google. How to develop conversation app quickly?”

This five-step development guide help to get hands-on experience with simple conversation assistant, like from buiding concept, development, and its integration.

Screen Shot 2017-05-23 at 12.52.54 PM
Developer Console:  Design, test, tune – all at one place

Easy to Test:

“Ok Google. How to preview and test actions?”

Google provides a web simulator which lets us preview actions that built in API.AI or the Actions SDK in an easy-to-use interface with debugging and voice input. It helps we make sure that Conversation Actions actually sound conversational and let us use the device without having the hardware.
In addition to the simulator, we can also test on an actual device by launching a preview version of actions built.

Easy to Deploy:

“Ok Google. Let’s deploy the conversation action.”

After setting up Actions on Google integration, we can deploy agent so that it is live and available for use by other users. We just need to first set up a Google App Engine project and register Conversation Action with Google.

So what are you waiting for? imitate some intelligence and “Hey machines, let’s talk! ;)”

Here at Clairvoyant, we’ve had a lot of fun learning these new conversational interactions and looking forward to building some cool intelligence. This post should be enough to get your interest towards something newly intelligent, get you up and running on creating your own cool new services. If you have any questions, feel free to leave a comment below. While believing in improving lives through design and technology, would also like to hear you if you have something interesting to share on this.

Wish you a happy time imitating your intelligence into first Google Action and getting introduced to API.AI.

Build you AI assistant with API.AI
Get started with Google Actions

Internet of Things,Big Data, Cloud Computing : The Perfect Match

What Is Internet of Things and How Does It Work?

Big Data is quickly becoming the next big asset for many organizations.  It would not be surprising for organizations to begin selling data of all types, including the metrics, knowledge, and insights gained from the data accumulated and analyzed. Riding on this wave of Big Data is Internet of Things (IoT).

Technology, along with low storage costs, is making it easy for organizations to process large amounts of data; as a result, a new trend is emerging known as “capture it all.”  Capture it all means collecting as much data about your customers’ product usage and behaviors as possible because the data collected may be useful in the future.  At a time when organizations are seeing the benefits of Big Data, IoT provides innovative ways of capturing data that can enhance these benefits.

Internet of Things is the concept that things (animals, people or objects) with a unique identifier can automatically transfer data over a network without human interaction.   There are a number of devices that can be connected to the Internet to create a network of ‘things’ that communicate with each other to make intelligent decisions.  This is nothing new; the concept of Ubiquitous computing and sensor networks has been in use for a long time.

Why the Buzz Now?

Four technology trends are fueling the IoT revolution and renewing interest, they include:

  1. Big Data: Big Data’s success is making people realize the value of data, including the ways to identify valuable insights from data once considered junk.   IoT deployments can produce huge amounts of data, as sensors are constantly sensing stimuli and triggering real-time events.  It becomes relatively easy for the data to get accumulated over-time and the big data ecosystem or platform makes it easy to process these huge amounts of data.
  2. Cloud Computing: With the advent of Cloud computing, computing power and data storage has become cheaper and easier to store and process data in Cloud.
  3. Ubiquitous connectivity: With the increase in the usage of smart phones with data plans along with the demand for connectivity to the Internet over smart phones, the infrastructure has been upgraded. Many IoT architectures are piggy backing on connectivity of the smart phone.
  4. Low cost sensors: The cost of Wi-Fi sensors and devices is in gradual decline. Standards like Near Field Communication (NFC), and iBeacon are becoming mainstream and supported by smart phones. As a result, App Developers can use them to creatively build IoT use cases. These improvements in the Bluetooth technology, Bluetooth low energy or BLE, are also becoming a catalyst to the IoT revolution.  In addition to the above, the ability of improved sensors to discreetly capture data is also a stimulus.

How Internet of Things Is Used Today

Internet of Things deployments implemented right have the potential to become Big Data’s killer Application.  Refer to figure 1 for the architecture used for typical IoT deployments.



Sensors that allow sensing of events are delivered to the mother ship on the cloud via servers connected to the Internet.  Data from these sensors is communicated via BLE and temporarily stored on the smartphone.  The use cases and potential for IoT and Big Data is endless as well as incredible.

A number of products based on IoT are getting launched and also receiving overwhelming response and adoption.  Wearable devices like Fitbit, Basis, Smartwatch from Samsung, and Qualcomm are playing an increasing part in IoT awareness.   Google became an early adaptor of IoT with its’ acquisition of the Nest Labs – Smart Thermostat.  Also, Google is gradually getting into home automation with their set-top boxes, NEST, Google Fiber, and Smart watches powered by Android OS.  Currently devices like Fitbit continuously and discreetly capture activity levels and sleep quality and then transmit the data to cloud using the smart phone. The sensors in turn communicate the data to smartphone through BLE.  Just imagine if there is a way to start tracking blood pressure, anxiety levels, stress levels, and heart rate in a similar discreet manner.  We can have personal data that is collected about ones self and then used by physicians in predicting a change in daily routine that can be causing current health issues.  For example, if a patient is unable to sleep properly, having the data collected historically could be invaluable in predicting what could have caused the current problem.  Tools like GOOGLE Nest Smart thermostat, Smart Smoke and CO alarm constantly track or monitor the environment in the house including information about lighting, humidity, daily behavior of the home’s residents, temperature, and air quality.  Smartphone apps like Easily Do also discreetly record day to day activities of the owner by simply keeping track of the GPS on the cell phone.

Potential Uses for Internet of Things

Imagine a world where devices can talk to each other, communicate and exchange information, and make intelligent decisions based on the data collected. For example, if you are coming home from a workout and, based on the data from your Fitbit and Easily Do, information is communicated to Google Nest thermostat that you would be home in a few minutes.  This information can be used to make your house more comfortable and cooler upon your arrival.  Taking this example a little further. What if data is captured from many people and made available to researchers (After anonymizing of Personal Identification Information).  This captured data can provide researchers and scientists with valuable data to study and find correlations between activities/actions that cause people to be susceptible to diseases.  This data can then provide feedback to users who may be susceptible to a disease and allow preventive measures to begin. In addition, trends/patterns may be identified that enable researchers to identify the reasons for diseases like heart attacks and Parkinson’s.  In addition, many of us may have seen sci-fi movies where an Artificial Intelligence system talks back and gives advice by analyzing a situation. Those days may not be far off due to the way technology trends in regards to IoT, Cloud computing, and Big Data are coming together.

Internet of Things’ Future

Gartner predicts that Internet of Things will affect every industry.  As a result, finding top analytics talent qualified to manage massive amounts of data will be difficult in the years ahead.  A yearlong research project conducted by Accenture shows that the United States is projected to create nearly 39,000 new jobs for analytics experts through 2015.  Only 23 percent if these jobs will be filled by qualified candidates. Cisco’s CEO, John Chambers, predicts that during the next decade the impact of Internet of Things will be 5 to 10 times greater than the Internet was on society and believes that IoT opens up a $19 Trillion opportunity during the same period.

IoT is here to stay and will make Big Data even bigger.  Our challenge, as IT professionals, is to discover innovative ways to use this technology that will enhance the general population’s lifestyle as well as benefit companies bottom line.