Agile Champs

Just a recap, in my previous post, I had primarily focused on the Algorithms that are at the core of the ML/DL space and how humans are at the helm of this impact of machines. This will be the concluding post for this series where we will discuss the applications/organizations who have successfully implemented the AI/ML frameworks and how they are benefiting out of it.

Have you ever thought how Google Maps predicts the traffic so accurately or how Amazon recommends products for you or even how self-driving cars work? If yes, then let us see the top 8 applications of Machine Learning. 

1. Google Maps Traffic Prediction

We will start with one of the applications of Machine Learning that we use in our day-to-day life, i.e., Google Maps’ traffic prediction. Google Maps is very accurate in predicting traffic. Google uses the information from the phone the app is installed upon to calculate how many cars are there on the road or how fast they are moving. Also the more people are on-boarded on this app, the traffic data becomes more accurate.

It also has incorporated traffic data from an app called ‘Waze’ to monitor traffic reports from local transportation departments and also keeps a history of traffic patterns on specific roads so its prediction is far more accurate. So, this was about Google Maps using a Machine Learning algorithm to analyze and predict the result using your data. More data you feed more accurate it becomes!

2. Google Translate

Google Translate enables us to translate documents, sentences, and websites instantly. All these translations come from computers that use statistical machine translation. For teaching someone a new language, we usually start off by teaching the vocabulary, grammatical rules, and then explain about constructing sentences but rules here contain a lot of exceptions. When you combine all of these exceptions in a computer program, then the quality of the translation begins to breakdown. Hence Google Translate, took a slightly different approach. Instead of teaching every rule of a language to the computer, what it does is, it lets the computer find the rules by itself. Google Translate does this with the help of Machine Learning. This is done by examining billions of documents that are already translated by human translators. Google Translate collects text from multiple sources. After the text or the data is collected, the machine tries to find patterns by scanning the text. Once the machine detects the pattern, this pattern is used multiple times for translating similar text. Repetitions of the same process by the machine will detect millions of patterns that will make the machine a perfect translator. Google’s translation is undoubtedly perfect, but by constantly providing newly translated text, it can get smarter and translate better. This is how Google translates your speech.

Now, we will move on to the applications of Machine Learning by looking at Facebook’s Automatic Alt Text.

3. Facebook’s Automatic Alt Text

Facebook’s Automatic Alt Text is one of the wonderful applications of Machine Learning for the blind. Facebook has rolled out this new feature that lets the blind users explore the Internet. It is called Automatic Alternative Text. With the help of this, the blind are getting the tools by which they can experience the outside world and the Internet. Blind people use screen readers that help in describing websites or apps. Facebook has estimated that there are more than a billion photos shared every day. However, the pictures shared would be of no use for the blind if they don’t come up with the text that outlines the picture. So, Facebook is resolving this problem with the help of ‘Automatic Alt Text.’ Here, when the built-in reader is turned on and when we tap on a picture, then Facebook’s Machine Learning algorithms try to recognize the features of the image and then create an alt text. This alt text will describe the picture with the help of the screen reader.

Recently, Twitter has also added a feature that makes use of alt text for images.

This was all about the applications of Machine Learning which Facebook developed to help the blind people experience the world.

Further, in this blog on ‘Applications of Machine Learning,’ we will see another application of Machine Learning, that is, Amazon’s recommendation engine.

4. Amazon’s Recommendation Engine

Amazon uses Machine Learning with Big Data to power its recommendation engine. It involves three stages: events, ratings, and filtering.

In the events phase, Amazon tracks and stores data regarding customer behavior and their activities on the site. Every click the user makes is an event, and the record of the user is logged in the database. This way, different types of events are captured for different kinds of actions like a user liking a product, adding a product to the cart, or purchasing a product.

Next phase is ratings. Ratings are important as they reveal what the user feels about the product. The recommendation system then assigns implicit values on different kinds of user actions like four-star for purchase, three-star for like, and two-star for a click, and so on.

Amazon’s recommendation system also uses Natural Language Processing to analyze the feedback which is provided by the user. The feedback can be something like ‘the product was great but the packaging was not good at all.’ With the help of Natural Language Processing, the recommendation system calculates the sentiment score and then classifies the feedback as positive, negative, or neutral.

Now, the last phase is filtering. In this step, the machine filters the product based on the ratings and other user data. The recommendation system uses different kinds of filtering such as collaborative filtering, user-based filtering, and hybrid filtering.

Collaborative filtering is one in which all the users’ choices are compared and they get a recommendation. For example, a user X likes products A, B, C, and D, and the user Y likes products A, B, C, D, and E. So, there is a chance that the user X will also like the product E, and the machine will recommend the product E to the user X.

After that comes the user-based filtering. In this, the users’ browsing history such as likes, purchasing, and ratings are taken into account before providing the recommendation.

Finally, in hybrid filtering, there is a mix and match of both the collaborative and the user-based filtering.

So, this is how Amazon recommends products for you. The applications of Machine Learning are not limited to just Amazon; organizations such as Alibaba, eBay, and Flipkart also use the same approach.

Going ahead in this blog on ‘Applications of Machine Learning,’ we will see about spam detection in Gmail.

5. Spam Detection in Gmail

Spam detection is the most commonly used mechanism in our day today life that makes use of filters. Algorithms are regularly updated based on the new potential threads found, advancement in technology, and the reaction given by users to spammed mails. Spam filters remove the threats using text filters based on the sender’s history. So, in this blog on applications of Machine Learning, next, we will see text filters, client filters, and spam filters.

That’s the outline of spam detection and how Gmail understands which email is spam. However, the real-time processes are a lot more complex and it consumes a lot of data. It is also used in fraud detection.

After spam detection in the applications of Machine Learning, we will move on to Amazon Alexa, which is another wonderful application of Machine Learning.

6. Amazon Alexa

The brain or voice of Echo is known as Amazon Alexa. It is capable of doing several tasks such as giving the weather report and playing your favorite song. Also, the word ‘Alexa’ is a wake word. As you say this word, it starts the recording of your voice. When you finish speaking, it sends the voice to Amazon. The service that persists this recording is called Alexa Voice Service or AVS, and it’s one of the magnificent applications of Machine Learning. This service is run by Amazon.

AVS interprets the command from the recorded audio. It is also called a voice detection service that can work with many other online services.

The commands interpreted by Alexa can be like asking for time and weather reports. After the command has been noted, it is sent to Amazon. Then, AVS gives the response by telling you the time and weather reports with the help of an audio file sent by Amazon servers.

You can also give some complex tasks such as if you tell Alexa to tell you the ‘Applications of Machine Learning,’ then AVS will search the keywords you have set up in servers.

Alexa can also control your home appliances by voice commands if you are using smart electronic devices such as Philips smart bulbs. You can give commands to Alexa to switch on or off the lights. You can even link it to Domino’s and order pizza by giving commands to Alexa. Isn’t it a magnificent application of Machine Learning?

Echo and Alexa can perform a lot of things. Amazon is continuously adding more skills to Alexa that will make it better.

Well, if you consider these products, Amazon is not the only company in the market that has used this application of Machine Learning. Google uses ‘Ok Google’ as its voice command services, Apple uses ‘Siri,’ and Microsoft uses ‘Cortana.’ Even if they are using the same approach, i.e., voice commands processed in cloud servers, they are not as good as Alexa.

Now, we will move on to another superb application of Machine Learning, i.e., self-driving cars!

7. Tesla’s Self-driving Cars

Tesla’s self-driving cars is another of the most-used applications of Machine Learning. A recent study has shown that over 90 percent of the road accidents are caused by human errors, and these mistakes are often catastrophic. The accidents have led to a massive number of unnecessary deaths; lives that could have been saved if they were driving safely. This is where the self-driving cars come into the picture. Thus, this real-world application of Machine Learning has led the automobile industry to a new and safer direction. These self-driven cars are autonomous cars that are safer than the cars driven by humans. Things that make these cars safe are that they are not affected by factors like illness or emotions of the driver.

Self-driving cars persistently observe the environments and scan all the directions and make their move. Due to their mechanism of not lagging in observation, self-driving cars work perfectly.

Working of Self-driving Cars

Self-driving cars are a real-world example of Machine Learning that mainly uses three different technologies: IoT sensors, IoT connectivity, and software algorithms.

Talking about self-driving cars is not limited to Tesla. In today’s world, the most famous self-driving cars are those made by Tesla and Google. Tesla cars work by examining the surroundings with the use of a software system that is the auto-pilot. As we use our eyes to visualize the surrounding world, the auto-pilot does it with the help of hi-tech cameras for recognizing objects. After that, it interprets the information and then makes the best conclusion out of it. This major application of Machine Learning is revolutionizing the automobile industry.

Next, in this blog on the applications of Machine Learning, we will look at the Netflix movie recommendation system.

8. Netflix Movie Recommendation

Netflix movie recommendation discovers 80% of the movie/TV shows that are streamed which means, the majority of what we decide to watch on Netflix is a result of the decisions made by its algorithm.

Netflix uses Machine Learning algorithms to recommend the list of movies and shows that you might have not initially chosen. To do this, it looks at threads within the content.

There are three legit tools for Netflix, and they are as follows: the first is Netflix members, the second is taggers who understand everything about the content, and the third is the Machine Learning algorithms that take all of the data and put them together.

Netflix uses different kinds of data from these profiles. It keeps track of what you guys are watching from your profile, what you watch after completing your current video, and even what you have watched earlier. It also keeps track of what you have watched a year ago or what you are currently watching, or at what time of the day you are watching. So, this data is the first leg of the metaphorical tool.

Now, they are combining this information with more data to understand the content of the shows that you are watching. This data is gathered from dozens of in-house and freelance stuff watched every minute and every show on Netflix, and they tag them. All the tags and user behavior data are taken and fed into a very sophisticated Machine Learning algorithm that figures out what’s the most important.

Well, these three legit tools are used to analyze the taste of communities around the world. It’s about people who watch the same kind of things that you watch. Viewers are made to fit into thousands of different taste groups that affect recommendation pop-ups on their screen at the top, as an interface with joined rows of similar content.

Across the globe, the tags used are the same for all the Machine Learning algorithms. The data Netflix feeds into its algorithms can be broken down into two types: implicit and explicit.

Explicit data is what you literally tell. For example, you give thumbs up to Friends and Netflix gets it.

Implicit data is real behavioral data. It’s like, you did not explicitly tell Netflix that you like Black Mirror, but you just watched it in two nights. So, here Netflix understands the behavior. But, just as a matter of fact, the majority of useful data is implicit data.

There are a lot more real-world applications of Machine Learning, but those described in this blog are the major applications of Machine Learning. Now that we have seen various applications of Machine Learning which are revolutionizing the world. We are moving into the next generation with the full-fledged Machine Learning technology that will help in giving a whole new direction.

This is the last blog in the series on Machine Learning and the related space. Hope you all enjoyed reading through the posts as much as I enjoyed putting them together. Stay tuned while I come back with yet another series on a technology topic.

Please feel free to review my earlier series of posts on AI-ML Past, Present and Future – distributed across 8 blogs.

Just a recap, in my previous post, I had primarily focused on the tools/techniques/frameworks and hardware that are prominent in the implementation of an AI/ML framework for any organization. In this post, we are going to dwell a little deeper into the Algorithms that are at the core of the ML/DL space and how humans are at the helm of this impact of machines.

Algorithms are becoming an integral part of our daily lives. About 80% of the viewing hours on Netflix and 35% of the retail on Amazon are due to automated recommendations by the so called AI/ML engines. Designers in companies like Facebook know how to make use of notifications and gamifications to increase user engagement and exploit & amplify various human vulnerabilities such as social approval and instant gratification. In short, we are nudged and sometimes even tricked into making choices by algorithms that need to learn. In case of the products and services we buy, the information we consume and who we mingle with online, algorithms are playing an important role in practically every aspect of it.

In the world of AI, a new challenge that is being increasingly discussed is the biases that creep into algorithms. Due to these biases, when we leave it to algorithms to take decisions, there can be unintended consequences. More so, as algorithms deployed by tech companies are used by billions of people, its damage because of biases can be significant. Moreover, we have a tendency to believe that algorithms are predictable and rational. So we tend to overlook many of their side effects.

How today’s algorithms differ?

In the past, developing an algorithm involved writing a series of steps that a machine could implement repeatedly without getting tired or making a mistake. In comparison, today’s algorithms, based on machine learning, do not follow a programmed sequence of instructions but ingest data and figure out for themselves the most logical sequence of steps and then keep working on improvement as they consume more and more data.

Machine learning itself is more sophisticated as traditional (supervised) ML, a programmer usually specifies what patterns to look for. The performance of these methods improves as they are exposed to greater data but this is limited. In deep learning, programmers do not specify what patterns to look for but the algorithm evaluates the training data in different ways to identify patterns that truly matter. Human beings may not identify some of these patterns.

Deep learning models contain an input layer of data and an output layer of the desired prediction and the multiple hidden layers in between that combine patterns from previous layers to identify abstract and complex patterns in the data. For instance

Unlike traditional algorithms, the performance of deep learning algorithms keeps improving as more data is fed.

Decision making and avoiding unintended consequence

AI involves enabling computers to do the tasks that human beings can handle. This means computers must be able to reason, understand language, navigate the visual world and manipulate objects. Machine learning enhances this by learning from experience. As algorithms become more and more sophisticated and develop newer capabilities, they are going beyond their original role of decision support to decision making. The flip side is that as algorithms become more powerful, there are growing concerns about their opaqueness and unknown biases. The benefits of algorithms seem to far outweigh the small chance of an algorithm going rogue now and then. It is important to recognize that while algorithms do an exceptionally good job of achieving what they are designed to achieve, they are not completely predictable. They do have side effects like some medicines. These consequences are of three types

Perverse results affect precisely what is measured and have a better chance of being detected. Unexpected drawbacks do not affect the exact performance metrics that are being tracked. and difficult to avoid them. Facebook’s Trending Topics algorithm is a good example. The algorithm ensured that the highlighted stories were genuinely popular. But it failed to question the credibility of sources and inadvertently promoted fake news. So inaccurate and fabricated stories were widely circulated in the months leading up to the 2016 US Presidential elections. The top 20 false stories in that period received greater engagement on Facebook than the top 20 legitimate ones.

Content and Collaborative filtering systems

Content based recommendation systems start with detailed information about a product’s characteristics and then search for other products with similar qualities. Thus, content based algorithms can match people based on similarities in demographic attributes- age, occupation, location, shared interests and ideas discussed on social media.

Collaborative filtering recommendation algorithms do not focus on the product’s characteristics. These algorithms look for people who use the same products that we do. For example, two of us may not be connected on Linked In but if we have more than a hundred mutual connections, we will get a notification that we should perhaps get connected.

Digital Framework

Algorithms also leverage the principle of digital neighborhood. One of the earliest pioneers of this principle was Google. In the late 1990s, when the internet was about to take off, the most popular online search engines relied primarily on the text content within web pages to determine their relevance. If a lot of other sites have a link to our website, then our website must be worth reading. It is not what we know but how many people know us that gets our website higher rankings. Research reveals that when Oprah Winfrey recommends a book, sales increase significantly but the books recommended by Amazon also get a significant boost. That is why digital neighborhoods are so important. For products that are recommended on many other product pages, recommendation algorithms drive a dramatic increase in sales. Spotify initially used a collaborative filter but later combined it with a content based method.

The predictability

AI began with Expert systems, ie systems which capture the knowledge and experience of experts. These systems suffer from two drawbacks.

• Do not automate the decision making process.
• Can’t code a response to every kind of situation.

We can either create intelligent algorithms in highly controlled environments, expert systems style to ensure they are highly predictable in behavior. But these algorithms will encounter problems they were not prepared for. Alternatively, we can use machine learning algorithms to create resilient but also somewhat unpredictable algorithms. This is the predictability – resilience paradox. Much as we may desire fully explainable and interpretable algorithms, the balance between predictability and resilience inevitably seems to be tilting in the latter direction.

Technology is most useful when it helps human beings to solve the most sophisticated problems which involves creativity. To solve such problems, we will have to move away from predictable systems.  One solution to resolve the predictability resilience paradox is to use multiple approaches. Thus, in a self-driving car, machine earning might drive the show but in case of confusion about a road sign, a set of rules might tick in.

Environmental Factors that Support

Human behavior is shaped both by hereditary and environmental factors. Same is the case with algorithms. There are three components we need to consider:

While data, algorithms and people play a significant role in determining the outcomes of the system, the sum is greater than the parts. The complex interactions among the various components have a big impact.

Need Trust

Many professions will be redefined if algorithmic systems are adopted intelligently by users. But if there are public failures, we cannot take user adoption for granted. We might successfully build intelligent diagnostic systems and effective driver-less cars but in the absence of trust, doctors and passengers will be unwilling to use them. So it is important to increase trust in algorithms. Otherwise, they will not gain acceptance. According to some estimates, driver-less cars would save up to 1.5 million lives just in the US and close to 50 million lives globally in the next 50 years. Yet, in a poll conducted in April 2018, 50% of the respondents said they considered autonomous cars less safe than cars driven by human beings.

Rules and Regulations

Decision making is the most human of our abilities. Today’s algorithms are advancing rapidly into this territory. So we must develop a set of rights, responsibilities and regulations to manage and indeed thrive in this world of technological innovations. Such a bill of rights should have four components:

AI systems have mastered games – now it’s time to master reality! Sometime back, Facebook developed two bots and trained them on negotiation skills. The bots were exposed to thousands of negotiation games and taught how conversations would evolve in a negotiation and how they should respond. The outcome of this training far exceeded expectations. The bots learnt how to trade items, developed their own short hand language. When the bots were made to negotiate with human beings again, the people on the other side did not even realize this!

Stay tuned…. Part 5 of this foray, we will look into the details of top 8 examples of organizations that have successfully implemented the AI/ML framework and how they are benefiting out of it.

Please feel free to review my earlier series of posts on AI-ML Past, Present and Future – distributed across 8 blogs.

Just a recap, in my previous post, I had primarily looked upon amount, structure of data that needs to be served as input to machine learning and the process used for its implementation. In this post, we are going to continue to learn more about the tools/techniques and hardware that will be required for these processes to excel.

While the majority of us are appreciating the early applications of machine learning, it continues to evolve at quite a promising pace, introducing us to more advanced algorithms like Deep Learning. Why DL is so good? It is simply great in terms of accuracy when trained with a huge amount of data. Also, it plays a significant role to fill the gap when a scenario is challenging for the human brain. So, quite logical this contributed to a whole slew of new frameworks appearing. Please see below the top 10 frameworks that are available and their intended use in the relevant industry space.

Just to provide a comprehensive analysis of the best tools that would undoubtedly recommend. So what is the final advice?

The speed of your algorithm is dependent on the size and complexity of your data, the algorithm itself, and your available hardware. In this section, we’re going to focus on hardware considerations when:

• Training the model
• Running the model in production

Some things to remember:

If you need results quickly, try machine learning algorithms first. They are generally quicker to train and require less computational power. The main factor in training time will be the number of variables and observations in the training data.

Deep learning models will take time to train. Pre-trained networks and public datasets have shortened the time to train deep learning models through transfer learning, but it is easy to underestimate the real-world practicalities of incorporating your training data into these networks. These algorithms can take anywhere from a minute to a few weeks to train depending on your hardware and computing power. 

Training the Model

Desktop CPUs: are sufficient for training most machine learning models but may prove slow for deep learning models.

CPU Clusters: Big data frameworks such as Apache Spark™ spread the computation across a cluster of CPUs.

The cluster or cloud option has gained popularity due to the high costs associated with obtaining the GPUs, since this option lets the hardware be shared by several researchers. Because deep learning models take a long time to train (often on the order of hours or days), it is common to have several models training in parallel, with the hope that one (or some) of them will provide improved results.

GPUs: are the norm for training most deep learning models because they offer dramatic speed improvements over training on a CPU. To reduce training time it is common for practitioners to have multiple deep learning models training simultaneously (which requires additional hardware).

Running the Model in Production

The trend toward smarter and more connected sensors is moving more processing and analytics closer to the sensors. This shrinks the amount of data that is transferred over the network, which reduces the cost of transmission and can reduce the power consumption of wireless devices.

Several factors will drive the architecture of the production system:

• Will a network connection always be available?
• How often will the model need to be updated?
• Do you have specialized hardware to run deep learning models?

Will a network connection always be available?

Machine learning and deep learning models that run on hardware at the edge will provide quick results and will not require a network connection.

How often will the model need to be updated?

Powerful hardware will need to be available at the edge to run the machine learning model, and it will be more difficult to push out updates to the model than if the model resided on a centralized server.

Tools are available that can convert machine learning models, which are typically developed in high-level interpreted languages, into standalone C/C++ code, which can be run on low-power embedded devices.

Do you have specialized hardware to run deep learning models?

For deep learning models, specialized hardware is typically required due to the higher memory and compute requirements.

We have now looked through the different frameworks/tools and techniques that are prevalent in the industry for Machine and Deep Learning initiatives in your organization. We also looked through the recommendation on which framework works for your specific request. Also we looked at the Hardware that is required to run these models Or to make the frameworks active.

Stay tuned…. Part 4 of this foray, we will look into Usage of all these Machine and deep learning algorithms, models and frameworks that are prevalent in the industry.

Please feel free to review my earlier series of posts on AI-ML Past, Present and Future – distributed across 8 blogs.