100% found this document useful (1 vote)
199 views

Internship Project.1

1. Machine learning is a type of algorithm that allows software to become more accurate at predicting outcomes without being explicitly programmed. It works by building models from input data that can be used to predict future events based on patterns in the training data. 2. Modern AI is used in many applications like face tagging, object detection, navigation apps, and translation tools. For example, Google Directions uses machine learning techniques to analyze traffic in real-time and provide the fastest route. 3. Early AI development relied mainly on statistical techniques like regression, classification, clustering, and decision trees applied to limited datasets. Today, deep learning allows machines to learn from vast amounts of data using artificial neural networks modeled after the human brain.

Uploaded by

Naveen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
100% found this document useful (1 vote)
199 views

Internship Project.1

1. Machine learning is a type of algorithm that allows software to become more accurate at predicting outcomes without being explicitly programmed. It works by building models from input data that can be used to predict future events based on patterns in the training data. 2. Modern AI is used in many applications like face tagging, object detection, navigation apps, and translation tools. For example, Google Directions uses machine learning techniques to analyze traffic in real-time and provide the fastest route. 3. Early AI development relied mainly on statistical techniques like regression, classification, clustering, and decision trees applied to limited datasets. Today, deep learning allows machines to learn from vast amounts of data using artificial neural networks modeled after the human brain.

Uploaded by

Naveen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 32

1.

Machine Learning – Introduction

We have seen Machine Learning as a buzzword for the past few years, the reason for this might
be the high amount of data production by applications, the increase of computation power in the
past few years and the development of better algorithms.Machine Learning is used anywhere from
automating mundane tasks to offering intelligent insights, industries in every sector try to benefit
from it. You may already be using a device that utilizes it. For example, a wearable fitness tracker
like Fitbit, or an intelligent home assistant like Google Home. But there are much more examples
of ML in use.

1.1 What is Machine Learning?

Machine learning (ML) is a category of an algorithm that allows software applications to become
more accurate in predicting outcomes without being explicitly programmed. The basic premise of
machine learning is to build algorithms that can receive input data and use statistical analysis to
predict an output while updating outputs as new data becomes available.

1
2. Machine Learning – What Today’s AI Can Do?

When you tag a face in a Facebook photo, it is AI that is running behind the scenes and
identifying faces in a picture. Face tagging is now omnipresent in several applications that
display pictures with human faces. Why just human faces? There are several applications that
detect objects such as cats, dogs, bottles, cars, etc. We have autonomous cars running on our
roads that detect objects in real time to steer the car. When you travel, you use Google Directions
to learn the real-time traffic situations and follow the best path suggested by Google at that point
of time. This is yet another implementation of object detection technique in real time.

Let us consider the example of Google Translate application that we typically use while visiting
foreign countries. Google’s online translator app on your mobile helps you communicate with
the local people speaking a language that is foreign to you.
There are several applications of AI that we use practically today. In fact, each one of us use AI
in many parts of our lives, even without our knowledge. Today’s AI can perform extremely
complex jobs with a great accuracy and speed. Let us discuss an example of complex task to
understand what capabilities are expected in an AI application that you would be developing
today for your clients.
Example:

We all use Google Directions during our trip anywhere in the city for a daily commute or even
for inter-city travels. Google Directions application suggests the fastest path to our destination at
that time instance. When we follow this path, we have observed that Google is almost 100%
right in its suggestions and we save our valuable time on the trip.
You can imagine the complexity involved in developing this kind of application considering that
there are multiple paths to your destination and the application has to judge the traffic situation in
every possible path to give you a travel time estimate for each such path. Besides, consider the
fact that Google Directions covers the entire globe. Undoubtedly, lots of AI and Machine
Learning techniques are in-use under the hoods of such applications.
Considering the continuous demand for the development of such applications, you will now
appreciate why there is a sudden demand for IT professionals with AI skills.

2
3. Machine Learning – Traditional AI

The journey of AI began in the 1950's when the computing power was a fraction of what it is
today. AI started out with the predictions made by the machine in a fashion a statistician does
predictions using his calculator. Thus, the initial entire AI development was based mainly on
statistical techniques.
In this chapter, let us discuss in detail what these statistical techniques are.

Statistical Techniques

The development of today’s AI applications started with using the age-old traditional statistical
techniques. You must have used straight-line interpolation in schools to predict a future value.
There are several other such statistical techniques which are successfully applied in developing
so-called AI programs. We say “so-called” because the AI programs that we have today are
much more complex and use techniques far beyond the statistical techniques used by the early AI
programs.

Some of the examples of statistical techniques that are used for developing AI applications in
those days and are still in practice are listed here:

 Regression
 Classification
 Clustering
 Probability Theories
 Decision Trees

Here we have listed only some primary techniques that are enough to get you started on AI
without scaring you of the vastness that AI demands. If you are developing AI applications based
on limited data, you would be using these statistical techniques.

3
However, today the data is abundant. To analyze the kind of huge data that we possess statistical
techniques are of not much help as they have some limitations of their own. More advanced
methods such as deep learning are hence developed to solve many complex problems.
As we move ahead in this tutorial, we will understand what Machine Learning is and how it is
used for developing such complex AI applications.

4
4 . Machine Learning – Categories of Machine Learning

Machine Learning is broadly categorized under the following headings:

Machine learning evolved from left to right as shown in the above diagram.
 Initially, researchers started out with Supervised Learning. This is the case of housing
price prediction discussed earlier.

 This was followed by unsupervised learning, where the machine is made to learn on its
own without any supervision.

 Scientists discovered further that it may be a good idea to reward the machine when it
does the job the expected way and there came the Reinforcement Learning.

 Very soon, the data that is available these days has become so humongous that the
conventional techniques developed so far failed to analyze the big data and provide us the
predictions.

 Thus, came the deep learning where the human brain is simulated in the Artificial Neural
Networks (ANN) created in our binary computers.

 The machine now learns on its own using the high computing power and huge memory
resources that are available today.

 It is now observed that Deep Learning has solved many of the previously unsolvable
problems.

5
Let us now study each of these categories in more detail.

4.1 Supervised Learning


Supervised learning is analogous to training a child to walk. You will hold the child’s hand,
show him how to take his foot forward, walk yourself for a demonstration and so on, until the
child learns to walk on his own.
4.1.1Regression
Similarly, in the case of supervised learning, you give concrete known examples to the computer.
You say that for given feature value x1 the output is y1, for x2 it is y2, for x3 it is y3, and so on.
Based on this data, you let the computer figure out an empirical relationship between x and y.
Once the machine is trained in this way with a sufficient number of data points, now you would
ask the machine to predict Y for a given X. Assuming that you know the real value of Y for this
given X, you will be able to deduce whether the machine’s prediction is correct.
Thus, you will test whether the machine has learned by using the known test data. Once you are
satisfied that the machine is able to do the predictions with a desired level of accuracy (say 80 to
90%) you can stop further training the machine.
Now, you can safely use the machine to do the predictions on unknown data points, or ask the
machine to predict Y for a given X for which you do not know the real value of Y. This training
comes under the regression that we talked about earlier.
4.1.2 Classification
You may also use machine learning techniques for classification problems. In classification
problems, you classify objects of similar nature into a single group. For example, in a set of 100
students say, you may like to group them into three groups based on their heights - short,
medium and long. Measuring the height of each student, you will place them in a proper group.
Now, when a new student comes in, you will put him in an appropriate group by measuring his
height. By following the principles in regression training, you will train the machine to classify a
student based on his feature – the height. When the machine learns how the groups are formed, it
will be able to classify any unknown new student correctly. Once again, you would use the test
data to verify that the machine has learned your technique of classification before putting the
developed model in production.

6
Supervised Learning is where the AI really began its journey. This technique was applied
successfully in several cases. You have used this model while doing the hand-written recognition
on your machine. Several algorithms have been developed for supervised learning. You will
learn about them in the following chapters.

7
4.2 Unsupervised Learning
In unsupervised learning, we do not specify a target variable to the machine, rather we ask
machine “What can you tell me about X?”. More specifically, we may ask questions such as
given a huge data set X, “What are the five best groups we can make out of X?” or “What
features occur together most frequently in X?”. To arrive at the answers to such questions, you
can understand that the number of data points that the machine would require to deduce a
strategy would be very large. In case of supervised learning, the machine can be trained with
even about few thousands of data points. However, in case of unsupervised learning, the number
of data points that is reasonably accepted for learning starts in a few millions. These days, the
data is generally abundantly available. The data ideally requires curating. However, the amount
of data that is continuously flowing in a social area network, in most cases data curation is an
impossible task.
The following figure shows the boundary between the yellow and red dots as determined by
unsupervised machine learning. You can see it clearly that the machine would be able to
determine the class of each of the black dots with a fairly good accuracy.

The unsupervised learning has shown a great success in many modern AI applications, such as
face detection, object detection, and so on.

8
4.3 Reinforcement Learning
Consider training a pet dog, we train our pet to bring a ball to us. We throw the ball at a certain
distance and ask the dog to fetch it back to us. Every time the dog does this right, we reward the
dog. Slowly, the dog learns that doing the job rightly gives him a reward and then the dog starts
doing the job right way every time in future. Exactly, this concept is applied in “Reinforcement”
type of learning. The technique was initially developed for machines to play games. The machine
is given an algorithm to analyze all possible moves at each stage of the game. The machine may
select one of the moves at random. If the move is right, the machine is rewarded, otherwise it
may be penalized. Slowly, the machine will start differentiating between right and wrong moves
and after several iterations would learn to solve the game puzzle with a better accuracy. The
accuracy of winning the game would improve as the machine plays more and more games.

The entire process may be depicted in the following diagram:

This technique of machine learning differs from the supervised learning in that you need not
supply the labelled input/output pairs. The focus is on finding the balance between exploring the
new solutions versus exploiting the learned solutions.

9
4.4 Deep Learning
The deep learning is a model based on Artificial Neural Networks (ANN), more specifically
Convolutional Neural Networks (CNN)s. There are several architectures used in deep learning
such as deep neural networks, deep belief networks, recurrent neural networks, and
convolutional neural networks.
These networks have been successfully applied in solving the problems of computer vision,
speech recognition, natural language processing, bioinformatics, drug design, medical image
analysis, and games. There are several other fields in which deep learning is proactively applied.
The deep learning requires huge processing power and humongous data, which is generally
easily available these days.
We will talk about deep learning more in detail in the coming chapters.

4.5 Deep Reinforcement Learning


The Deep Reinforcement Learning (DRL) combines the techniques of both deep and
reinforcement learning. The reinforcement learning algorithms like Q-learning are now
combined with deep learning to create a powerful DRL model. The technique has been with a
great success in the fields of robotics, video games, finance and healthcare. Many previously
unsolvable problems are now solved by creating DRL models. There is lots of research going on
in this area and this is very actively pursued by the industries.
So far, you have got a brief introduction to various machine learning models, now let us explore
slightly deeper into various algorithms that are available under these models.

10
5. Machine Learning – Supervised Learning

Supervised learning is one of the important models of learning involved in training machines.
This chapter talks in detail about the same.

5.1 Algorithms for Supervised Learning

There are several algorithms available for supervised learning. Some of the widely used
algorithms of supervised learning are as shown below:
 k-Nearest Neighbours
 Decision Trees
 Naive Bayes
 Logistic Regression
 Support Vector Machines

As we move ahead in this chapter, let us discuss in detail about each of the algorithms.

5.1.1 k-Nearest Neighbours


The k-Nearest Neighbours, which is simply called kNN is a statistical technique that can be used
for solving for classification and regression problems. Let us discuss the case of classifying an
unknown object using kNN. Consider the distribution of objects as shown in the image given
below:

The diagram shows three types of objects, marked in red, blue and green colors. When you run
the kNN classifier on the above dataset, the boundaries for each type of object will be marked as
shown below:

11
Now, consider a new unknown object that you want to classify as red, green or blue. This is
depicted in the figure below:

As you see it visually, the unknown data point belongs to a class of blue objects. Mathematically,
this can be concluded by measuring the distance of this unknown point with every other point in
the data set. When you do so, you will know that most of its neighbours are of blue color. The

12
average distance to red and green objects would be definitely more than the average distance to
blue objects. Thus, this unknown object can be classified as belonging to blue class.

The kNN algorithm can also be used for regression problems. The kNN algorithm is available as
ready-to-use in most of the ML libraries.

5.1.2 Decision Trees


A simple decision tree in a flowchart format is shown below:

You would write a code to classify your input data based on this flowchart. The flowchart is self-
explanatory and trivial. In this scenario, you are trying to classify an incoming email to decide
when to read it.

13
In reality, the decision trees can be large and complex. There are several algorithms available to
create and traverse these trees. As a Machine Learning enthusiast, you need to understand and
master these techniques of creating and traversing decision trees.

5.1.3 Naive Bayes


Naive Bayes is used for creating classifiers. Suppose you want to sort out (classify) fruits of
different kinds from a fruit basket. You may use features such as color, size and shape of a fruit,
For example, any fruit that is red in color, is round in shape and is about 10 cm in diameter may
be considered as Apple. So to train the model, you would use these features and test the
probability that a given feature matches the desired constraints. The probabilities of different
features are then combined to arrive at a probability that a given fruit is an Apple. Naive Bayes
generally requires a small number of training data for classification.

5.1.4 Logistic Regression


Look at the following diagram. It shows the distribution of data points in XY plane.

From the diagram, we can visually inspect the separation of red dots from green dots. You may
draw a boundary line to separate out these dots. Now, to classify a new data point, you will just
need to determine on which side of the line the point lies.

14
5.1.5 Support Vector Machines

Look at the following distribution of data. Here the three classes of data cannot be linearly
separated. The boundary curves are non-linear. In such a case, finding the equation of the curve
becomes a complex job.

The Support Vector Machines (SVM) comes handy in determining the separation boundaries in
such situations.

15
6. Machine Learning – Unsupervised Learning
So far what you have seen is making the machine learn to find out the solution to our target. In
regression, we train the machine to predict a future value. In classification, we train the machine
to classify an unknown object in one of the categories defined by us. In short, we have been
training machines so that it can predict Y for our data X. Given a huge data set and not
estimating the categories, it would be difficult for us to train the machine using supervised
learning. What if the machine can look up and analyze the big data running into several
Gigabytes and Terabytes and tell us that this data contains so many distinct categories?

As an example, consider the voter’s data. By considering some inputs from each voter (these are
called features in AI terminology), let the machine predict that there are so many voters who
would vote for X political party and so many would vote for Y, and so on. Thus, in general, we
are asking the machine given a huge set of data points X, “What can you tell me about X?”. Or it
may be a question like “What are the five best groups we can make out of X?”. Or it could be
even like “What three features occur together most frequently in X?”.

This is exactly the Unsupervised Learning is all about.

6.1 Algorithms for Unsupervised Learning

Let us now discuss one of the widely used algorithms for classification in unsupervised machine
learning.

6.1.1 k-means clustering

The 2000 and 2004 Presidential elections in the United States were close — very close. The
largest percentage of the popular vote that any candidate received was 50.7% and the lowest was
47.9%. If a percentage of the voters were to have switched sides, the outcome of the election
would have been different. There are small groups of voters who, when properly appealed to,
will switch sides. These groups may not be huge, but with such close races, they may be big

16
enough to change the outcome of the election. How do you find these groups of people? How do
you appeal to them with a limited budget? The answer is clustering.

Let us understand how it is done.

 First, you collect information on people either with or without their consent: any sort of
information that might give some clue about what is important to them and what will
influence how they vote.

 Then you put this information into some sort of clustering algorithm.

Next, for each cluster (it would be smart to choose the largest one first) you craft a message that
will appeal to these voters.

 Finally, you deliver the campaign and measure to see if it’s working.
 Clustering is a type of unsupervised learning that automatically forms clusters of similar
things. It is like automatic classification. You can cluster almost anything, and the more
similar the items are in the cluster, the better the clusters are. In this chapter, we are going
to study one type of clustering algorithm called k-means. It is called k-means because it
finds ‘k’ unique clusters, and the center of each cluster is the mean of the values in that
cluster.

6.1.2 Cluster Identification


Cluster identification tells an algorithm, “Here’s some data. Now group similar things together
and tell me about those groups.” The key difference from classification is that in classification
you know what you are looking for. While that is not the case in clustering.

Clustering is sometimes called unsupervised classification because it produces the same result as
classification does but without having predefined classes. Now, we are comfortable with both
supervised and unsupervised learning.

17
7.Machine Learning – Skills for Machine Learning

Advanced analytics techniques, big data infrastructure and powerful algorithms are providing
organizations with the ability to utilize their data for significant business value. The challenge,
however, is finding the right skill sets and talent to make better use of the data.

The growth of data science requires a deeper set of skills and Capabilities from data science
practitioners. While the fundamentals of data science have been around for decades, only
recently have the tools and techniques matured to provide the capabilities necessary to
accomplish more advanced data analytics, AI and machine learning (ML) goals.

As such, there are many skills for machine learning and AI that data scientists, IT professionals
and ML engineers should hone before embarking on AI and ML projects. Some of these skills
are general math and statistics skills, while others are more specific to the technology and
platforms being used.

Statistics and math


Rather than programming their way to an explanation, data scientists need to derive insights
from raw data by using statistics, probability and various functions. Therefore, it should come as
little surprise that the fundamental competencies data scientists need are a core understanding of
statistics, probability and methods for data derivation.

7.1 Statistics and probability skills


The cornerstones of deriving insights from data are the mathematical areas of statistics and
probability. Advanced levels of statistics are the mainstay of data science and are applied
throughout the profession with data visualization, data modeling, identification of correlations,
regression, feature transformation, data imputation and dimensionality reduction, among others.

18
Data scientists require a firm grasp on concepts such as the following:

 Mean, median and mode;

 Standard deviation and variance;

 Correlation coefficients and the covariance matrix

 Probability distributions -- Binomial, Poisson, Normal;

 p-value;

 Bayes' Theorem; and

 Aspects of the confusion matrix including precision, recall, positive predictive value,


negative predictive value, receiver operating characteristic (ROC) curves, Central Limit
Theorem, R2 score, Mean Square Error, A/B testing and Monte Carlo Simulation.
7.2 Multivariable calculus and linear algebra
Linear algebra and multivariable calculus are widely applied by organizations in data science to
manipulate and transform data and derive insights. Linear algebra is applied in areas such as data
processing and transformation, dimensionality reduction and model evaluation. Core linear
algebra topics data scientists need to be familiar with include vectors, norms, matrices, matrix
transpositions and manipulations, dot products, eigenvalues and eigenvectors.

ML models, particularly deep learning approaches, rely on matrix math and multivariable


calculus. It's key that data scientists are familiar with multivariable calculus concepts such as
derivatives and gradients, step functions, sigmoid functions, logit functions, cost functions,
min/max values, Rectified Linear Unit functions and function plotting.

7.3 Optimization methods


In addition to core statistics and probability knowledge, data scientists need to understand how to
optimize functions, data and algorithms to achieve end objectives. Many ML algorithms that

19
focus on predictive applications achieve their goals by minimizing an objective function and
learning the weights applied to the testing data to obtain final predictions.

Other optimizations include the needs for cost and error functions, methods for rapidly
determining values from big data and iterating for better performance and accuracy. Key areas
data scientists should have knowledge in include cost function and objective functions,
likelihood and error functions, gradient descent algorithms and their variants.

7.3.1 Data analysis and wrangling


While being able to derive insights from data is the core of data science, the ability to present
that information in ways that can provide value to an organization is equally important.
Likewise, analysis of data requires access to sufficient volumes of structured data on which to
base those insights. As a result, data scientists also need to have key skills around data
visualization, data manipulation, data preparation and data wrangling.

7.4 Data visualization


Taking the numerical or classification insights from data and presenting them in a way that can
be understood by decision-makers is a vital skill for data scientists. Data visualization, which
embodies the concept of creating charts, graphs, diagrams and other illustrations of data, is
helpful for people who are better with visual information than numerical or quantified data. In
many ways, data visualization is a creative aspect of data science, and appeals to those with
design-thinking or UX priorities. The most important outcome of data visualization is
successfully building a story from the data using visualizations that people can easily understand.

Data scientists should have experience with a variety of data plotting and charting approaches
including the following:

 Histograms;

 Bar and area charts, pie and line charts, waterfall charts, thermometer and candlestick
charts;

 Segmentation and clustering diagrams;

20
 Scatter plots and bubble charts;

 Visualizations of classification space;

 Methods for visualization during exploratory data analysis;

 Frame and tree diagrams;

 Funnel Charts, Word Clouds, Heatmaps, Video And Image Annotations;

 Map And Geospatial Visualizations; And

 The use of a wide range of gauges, metrics and measures.

7.5 Data manipulation preparation and wrangling


Working with big data requires a lot of plumbing to get access to data in the right quantity and of
the right quality. While data engineering is its own field, data scientists need at least some core
knowledge and experience in how to access pools of big data and manipulate it into the shape
needed for analysis and processing. According to AI market intelligence firm Cognilytica (where
this writer is an analyst), over 80% of AI and data analytics project time is spent on data
wrangling and manipulation tasks.

One of the requirements for data wrangling is data access and collection. Data scientists should
have some experience in big data access through state-of-the-art and widely accepted data
platforms including Hadoop and Spark, as well as more traditional data access approaches
including SQL and NoSQL methods. They should also have experience with common databases
such as MongoDB and Postgres. In addition, data scientists should be familiar with manipulating
the data with data selection, data extraction and methods by which huge data sets are filtered to
the relevant portions.

In fact, 56% of data scientist positions list SQL as a requirement, according to a report from


Villanova University on the talent gap in data analytics.

Equally as important as data selection is dealing with questionable quality data. Collecting data
from multiple data sources can lead to many issues including missing, incorrect, conflicting or
potentially biased data. With data programming and collection approaches, data wrangling and

21
preparation requires addressing all of these forms of data imperfections and applying
transformations, manipulations, formatting changes and augmentation to improve overall data
quality.

Data scientists will need to know how to identify missing or erroneous data, methods of data
imputation, approaches for augmenting or enhancing data sets, methods for data transformation
and multiplication as needed, identification and treatment for outliers, correction of data types,
data scaling and normalization, detection of potential bias in data, data deduplication and data
anonymization.

7.5.1 Data analysis and modeling


With the right amount of data at the right level of quality and with the prerequisite understanding
of statistics and mathematical approaches for dealing with data, data scientists must apply those
skills to build models that an organization can use effectively for analysis and prediction. Data
scientists need to know how to create models, build analytics offerings and craft implementations
that are put into action by the organization.

7.6 Data analysis


Using open source as well as commercial offerings, data scientists need to know how to build
analytics products that can generate predictive, descriptive and projective results from data.
These models are built using existing data to generate results from future data. Data analysis
helps the organization apply its knowledge of data to new information to generate better insights
and provide stronger decision-making.

These models are generally relevant to the line of business and to organizational needs ranging
from recommending products to customers to projecting sales and inventory, from understanding
trends in customer or patient data to classifying a wide range of data into categories.

Data analysis is done using a wide range of tools including Excel, big data analysis tools like
Hadoop and Spark, commercial analytics offerings such as SAS and MATLAB and open source
offerings using R, Python, Java, Julia and other languages. Knowledge of these tools and using
them to achieve data analysis objectives is incredibly important to a data scientist's success.

22
7.7 ML algorithms, modeling and feature engineering
ML has become the most visible aspect of the modern data scientist's job as it requires them to
build models from data using their skills for machine learning methods and algorithms. Data
scientists need to understand the vast range of ML algorithms including:

 Decision Trees, Random Forests, Bagged And Boosted Tree Approaches;

 Bayesian Methods;

 K-Nearest Neighbors;

 Support Vector Machines;

 Ensemble Methods;

 Clustering Approaches Including K-Means, Gaussian Mixture And Principal Component


Analysis;

 Markov Models; And

 Recurrent Neural Networks, convolutional neural networks and Boltzmann machines.

Data scientists must ensure they are up to speed with the innovation taking place in ML
algorithms.

Additionally, data scientists need to understand how to perform model evaluation


and hyperparameter optimization. This means performing cross-validation and model
optimization steps, as well as understanding ROC and learning curves.

Data scientists aiming to have skills for machine learning will also need to know how to use
third-party models for their own needs to shorten overall model development time. This means
having an understanding of transfer learning and how to enhance models.

7.7.1 Platform and technology proficiencies


By putting all these pieces together, data scientists will also need to be proficient in
programming and technology to effectively do their job in their work environment.

23
7.8 Programming skills

Much of the technology around data science has evolved over the past few decades. Open source
as well as commercial offerings provide a plethora of tools, libraries, frameworks and support
functionality across the full lifecycle of the data scientist's job responsibilities.

Data scientists need proficiency in a range of languages including Python, R, Julia and Java-
based languages. Python in particular has been the star of the data science world. In 2018, 66%
of data scientists reported using Python every day, overtaking R as the most popular language for
data science. Julia and other languages are helpful for high-speed and big data processing, and
even the use of commercial offerings from SAS and MATLAB are helpful to accomplish a range
of data science and analytics tasks, particularly in enterprise settings where the ability to scale up
projects is important.

The more technical knowledge and skills data scientists have, the better.

7.9 Analytical and big data processing tools


Big data and data access technologies and tools are needed to extract meaningful insights from
big data. Data scientists should have some knowledge of platforms and frameworks for big data
processing including SQL, Spark, Hadoop, Hive and Pig. According to Villanova University's
report, 49% of data scientists ranked Apache Hadoop as the second most important skill for a
data scientist.

7.10 Cloud-based platforms and machine learning as a service (MLaaS)


Increasingly, much of the work of data science and ML engineering is done in the cloud. Data
scientists should have some experience with cloud-based MLaaS environments from Amazon,
Microsoft, Google and IBM, among others, with specific expertise in whichever of those
environments are utilized by their organization. Many of these platforms have a wide range of
tools, pretrained models and additional support for the full lifecycle of model development and
data science activities.

24
7.11 Data engineering and manipulation tools
To accomplish many of those data preparation and manipulation tasks, data scientists should
have experience using tools for big data manipulation, including open source offerings such as
Pandas, as well as those provided by commercial or cloud-based providers. Data scientists
should also have experience working with unstructured data such as images, videos, emails and
documents that come from different channels and sources.

25
8. Machine Learning Applications
Let’s explore other real-world machine learning applications that are sweeping the world! I’ll be
discussing the following Applications of Machine Learning one by one:

 Social Media Features

 Product Recommendations

 Image Recognition

 Sentiment Analysis

 Automating Employee Access Control

 Marine Wildlife Preservation

 Regulating Healthcare Efficiency and Medical Services

 Predict Potential Heart Failure

 Banking Domain

  Language Translation

Popular Machine Learning Applications

1. Social Media Features

Social media platforms use machine learning algorithms and approaches to create some attractive
and excellent features. For instance, Facebook notices and records your activities, your chats,
likes, and comments, and the time you spend on specific kinds of posts. Machine learning learns
from your own experience and makes friends and page suggestions for your profile.

26
2. Product Recommendations

One of the most popular and known applications of machine learning is Product
Recommendation. Product recommendation is one of the stark features of almost every e-
commerce website today, which is an advanced application of machine learning techniques.
Using machine learning and AI, websites track your behavior based on your previous purchase,
your searching pattern, your cart history, and make product recommendations.

27
3. Image Recognition

Image Recognition is one of the most significant and notable Machine Learning and AI
techniques: an approach for cataloging and detecting a feature or an object in the digital image.
This technique is being adopted for further analysis, such as pattern recognition, face detection,
or face recognition.

4. Sentiment Analysis

Sentiment analysis is a real-time machine learning application that determines the emotion or
opinion of the speaker or the writer. For instance, if someone has written a review or email (or
any form of a document), a sentiment analyzer will instantly find out the actual thought and tone
of the text. This sentiment analysis application can are used to analyze a review based website,
decision-making applications, etc.

28
5. Automating Employee Access Control

Organizations are actively implementing machine learning algorithms to determine the level of
access employees would need in various areas, depending on their job profiles.

6. Marine Wildlife Preservation

Machine learning algorithms are used to develop behavior models for endangered cetaceans and
other marine species, helping scientists regulate and monitor their populations.    

7. Regulating Healthcare Efficiency and Medical Services

Significant healthcare sectors are actively looking at using Machine Learning algorithms to
manage better. They predict the waiting times of patients in the emergency waiting rooms across
various departments of hospitals. The models use vital factors that help define the algorithm,
details of staff at various times of day, records of patients, and complete logs of department chats
and the layout of emergency rooms. Machine learning algorithms also come to play when
detecting a disease, therapy planning, and prediction of the disease situation. 

29
8. Predict Potential Heart Failure

An algorithm designed to scan a doctor’s free-form e-notes and identify patterns in a patient’s
cardiovascular history is making waves in medicine. Instead of a physician digging through
multiple health records to arrive at a sound diagnosis, redundancy is now reduced with
computers making an analysis based on available information.

9. Banking Domain

Banks are now using the latest advanced technology machine learning has to offer to help
prevent fraud & protect accounts from hackers. The algorithms determine what factors to
consider to create a filter to keep harm at bay. Various sites that are unauthentic will be
automatically filtered out and restricted from initiating transactions.

10. Language Translation

Machine learning plays a significant role in the translation of one language to another. We are
amazed at how the websites can translate from one language to another effortlessly and gives
contextual meaning as well. The technology behind the translation tool is called ‘machine
translation.’ It has enabled the world to interact with people from all corners of the world;
without it, life would not be as easy as it is now. It has provided a sort of confidence to travelers
and business associates to safely venture into foreign lands with the conviction that language will
no longer be a barrier.

30
9. Advantages and Disadvantages of Machine Learning

9.1 Advantages of Machine learning


1. Easily identifies trends and patterns

2. No human intervention needed

3. Continuous Improvement

4. Handling multi-dimensional and multi-variety data

5. Wide Applications

9.2 Disadvantages of Machine Learning


With all those advantages to its powerfulness and popularity, Machine Learning isn’t perfect.
The following factors serve to limit it:

1. Data Acquisition

2. Time and Resources

3. Interpretation of Results

4. High error-susceptibility

31
10. Future Scope and Conclusion

10.1 Future Scope


The scope of machine learning is not limited to a particular domain but it is expanding its wings all around. Either
it is the IT sector, healthcare domain, media, or entertainment field, the presence of machine learning can be felt
everywhere and some of the sectors like Automotive, Robotics plays a major role in the area of machine learning
and artificial Intelligence.Thus, The Future Scope Of Machine Learning will accelerate the processing power of
the automation system used in various technologies.

10.2 Conclusion
Getting to know some of the popular applications of machine learning along with technology
evolving at a rapid pace, we are excited about the possibilities which the Machine Learning
Course has to offer in the days to come. And as the demand for AI and machine learning has
increased, organizations require professionals with in-and-out knowledge of these growing
technologies and hands-on experience.

32

You might also like