Unit 4

SEMANTICS AND PRAGMATICS

contents
• Requirements for representation
• First Order logic
• Description Logics
• Syntax Driven Semantic analysis
• Semantic attachments
• Word senses, relations, thematic roles , selectional restrictions
• Word sense Disambiguation
• WSD using supervised
• Dictionary and Thesaurus
• Bootstrapping methods
• Word similarity using thesaurus and Distributional methods
Semantics
Definition:
Semantics is the study of meaning in language. It focuses on the literal meaning
of words, phrases, and sentences, independent of context.
Key Focus:
• Understanding what words mean and how those meanings combine.
• Analyzing concepts like synonyms, antonyms, word senses, and semantic
roles.
• Studying logical structures (e.g., "All cats are mammals").
Example :
Sentence: “The bank is by the river.”
• Semantics analyzes possible meanings of the word “bank” (financial
institution or riverbank).
• It tries to interpret the sentence based on dictionary definitions and word
relations.
Pragmatics
Definition:
Pragmatics is the study of how context influences meaning. It looks beyond literal meanings
to understand the speaker's intention, implied meanings, and how language is used in real
situations.
Key Focus:
• Understanding how meaning changes with context, tone, and background knowledge.
• Interpreting indirect speech, sarcasm, politeness, and conversational implicatures
(unstated implications).
• Analyzing deictic expressions (e.g., “here,” “now,” “that”) whose meaning depends on the
context.
Example in Language:
Sentence: “Can you pass the salt?”
• Semantics would treat it as a yes/no question asking about someone’s ability to pass the
salt.
• Pragmatics understands it as a polite request, not a literal question about capability.
Requirements for representation
• the meaning of language-related expressions can be captured
using formal structures, referred to as meaning representations.

• The frameworks that define the syntax and semantics of these

representations are known as meaning representation
languages.

• Meaning representations are crucial when linguistic inputs (like

words, grammar, and sentence structures) alone do not provide
enough information to understand or act on a task.
• This figure illustrates how the sentence "I have a car" can be represented
using various meaning representation approaches:
• First-Order Logic (FOL): Uses formal symbolic logic.
• Semantic Network: Represents relationships using nodes and edges (directed
graphs).
• Conceptual Dependency Diagram: Focuses on capturing the relationships
between concepts (like possession).
• Frame-Based Representation: Uses structured data with fields for specific
entities and their roles.
• All these approaches rely on symbols to represent objects, properties, and
relationships (e.g., speaker, car, possession) and link language to world
knowledge.
First Order Logic
• First-Order Logic (FOL) is a formal system used in mathematics,
philosophy, and computer science to express relationships,
properties, and facts about objects in the world. It extends
propositional logic by adding quantifiers, variables, and predicates to
represent more complex statements.

• In NLP, First-Order Logic (FOL) is used as a meaning representation

language to formalize the semantics (meaning) of natural language
sentences. It helps convert human language into a structured form
that computers can understand and reason with.
Components of First Order Logic:
Constants: Represent specific objects in the domain. Ex: "John", "Car",
"Apple".
Variables:Represent arbitrary objects. Ex: x, y, z.
Predicates: Describe properties of objects or relationships between
objects. Ex: Has(Car, Wheels) means "A car has wheels".
Quantifiers:
Universal Quantifier ( ∀ ): Means "for all" or "every".
Example 1: sentence :"All cars have wheels.“
FOL Representation: ∀x (Car(x) → HasWheels(x))
Explanation: "If x is a student, then x studies."
• Example 2: sentence : “All men are mortal”
FOL Representation : ∀x (Man(x) → Mortal(x))
i.e : If x is a man, then x is mortal
Existential Quantifier ( ∃ ) – "There Exists“ or “atleast one”
Example 1: sentence : Someone likes chocolate
FOL Representation : ∃x (Person(x) ∧ Loves(x, Chocolate))
i.e There exists a person x who loves chocolate.
Note:
• Lambda Notation : λx.Expression(x)
This represents an anonymous function that takes an input x and
returns Expression x as the output
Ex : Sentence: "John walks.“
Walks can be represented as: λx. Walk(x)
we apply john to the lambda expression for "walks":
(λx. Walk(x))(John) substitute x with john.
Walk(John)
Description Logics

• Description Logics aim to improve the understanding and specification

of semantics used in structured network representations.
• They provide a conceptual framework that is well-suited for domain
modeling.
• Description Logics belong to a family of logical approaches that
correspond to subsets of First-Order Logic (FOL).
Components of Description
Logics:
• Terminology (TBox): This contains knowledge about categories (also
known as concepts) and their relationships.

• Assertions (ABox): Contains facts about specific individuals belonging

to the categories defined in the TBox.

• Ontology: Organizes terminology hierarchically, capturing

subset/superset relationships between different concepts.
Subsumption in Description Logics
• Subsumption Relation (⊑):
• A hierarchical structure is established using subsumption relations between
concepts in a terminology.
• Subsumption is written as C ⊑ D, which is read as "C is subsumed by D". This
means:
• All members of category C are also members of category D.
• This relation is akin to a simple set relation where any element in C also belongs to D.
Example of Subsumption in the Restaurant Domain:
1.Restaurant ⊑ CommercialEstablishment:
1. This states that all restaurants are commercial establishments.
2.ItalianRestaurant ⊑ Restaurant
ChineseRestaurant ⊑ Restaurant
MexicanRestaurant ⊑ Restaurant
1. These assertions indicate that Italian, Chinese, and Mexican restaurants are
specific types (subtypes) of the broader category Restaurant.
Syntax Driven Semantic analysis
• Principle of Compositionality: The meaning of a sentence can be
derived from the meanings of its parts (words and phrases) and the
rules used to combine them.
• Pipeline Approach: Syntax-driven semantic analysis follows a pipeline
structure:
Syntactic Analysis: First, parse the sentence and create a syntactic
structure (parse tree).
Semantic Analysis: Next, interpret this syntactic structure to create a
meaningful representation of the sentence.
• The input goes through both syntactic and semantic analysis to
produce logical or meaningful output.
• The subtree for the verb "likes" generates a skeletal meaning
• The noun phrases ("Franco" and "Frasca") are combined with the verb template
to bind the appropriate variables in the verb’s semantic representation.
• Challenges:The process requires knowledge about how the verb’s arguments
(like subject and object) are linked to specific variables.
Since there can be infinite parse trees for a grammar, this makes the semantic
binding process complex.
• Solution is Rule to Rule hypothesis.
Semantic attachments
Word Senses, Relations, Thematic
Roles, and Selectional Restrictions
Relations between Word Senses:
Lexical relations capture the connections between word senses. Some
important types are:
• Synonymy: Words with similar meanings.
Example: Happy and Joyful.
• Antonymy: Words with opposite meanings.
Example: Hot vs. Cold.
• Hyponymy and Hypernymy:
• Hyponym: A more specific term.
Example: Rose is a hyponym of Flower.
• Hypernym: A more general term.
Example: Animal is a hypernym of Dog.
• Meronymy: A part-whole relationship.
Example: Wheel is a meronym of Car.
Thematic Roles (Semantic Roles):
Thematic roles describe the function that entities play in a sentence. Common
thematic roles include:
• Agent: The doer of an action.
Example: pretish (Agent) in “Pretish opened the door."
• Theme (Patient): The entity undergoing an action.
Example: The door (Theme) in “Pretish opened the door."
• Experiencer: The entity experiencing a psychological state.
Example: Mary (Experiencer) in "Mary is scared."
• Goal: The endpoint of an action.
Example: The park (Goal) in "They walked to the park."
• Instrument: The tool used to perform an action.
Example: The key (Instrument) in "John opened the door with a key."
• Source: The starting point of an action.
Example: Paris (Source) in "He traveled from Paris."
Selectional Restrictions:
• Definition:
Selectional restrictions are constraints on the types of arguments
(entities) that a verb can take, based on thematic roles and word
sense. They ensure semantic compatibility.
• Example:
• The verb eat typically selects for a subject capable of eating (like humans or
animals) and an object that is edible.
• Valid: John eats an apple.
• Invalid: The rock eats a chair. (This violates selectional restrictions.)
• Use in Natural Language Processing (NLP):
• Selectional restrictions help improve tasks like parsing, machine translation,
and semantic role labeling by filtering out nonsensical interpretations.
WSD using supervised
• Word Sense Disambiguation (WSD) is the task of determining the
correct sense of a word in a given context when the word has multiple
possible meanings.
In supervised WSD, machine learning models are trained on
annotated datasets, where each word occurrence is labeled with its
appropriate sense.
• Steps in Supervised WSD
Step 1. Data Collection and Annotation:
• Collect a dataset of sentences containing ambiguous words.
• Annotate each occurrence of the word with its correct sense.
Example: For the word bank, label its usage as either "financial
institution" or "river edge" depending on the sentence context.
Step 2. Feature Extraction:
Extract useful features from the sentences to help the model learn
the context in which different word senses appear.
Common features:
Bag-of-Words (BoW): Words surrounding the target word.
POS Tags: Part-of-speech tags of surrounding words.
Collocations: Frequent word pairs (e.g., "bank loan" indicates the
financial sense).
Syntactic features: Parse tree information or grammatical
relationships.
Word Embeddings: Vector representations of words using techniques
like Word2Vec or BERT.
Step 3: Training the Supervised Model: Train a machine learning
classifier on the labeled dataset.
Common classifiers for WSD:
• Naïve Bayes Classifier
• Decision Trees
• Support Vector Machines (SVMs)
• Neural Networks (e.g., LSTM, BERT)

Final step: Testing and Evaluation:

Evaluate the performance of the trained model using standard
metrics:
• Accuracy: Percentage of correctly disambiguated word senses.
• Precision, Recall, and F1-score: Useful for imbalanced datasets.
Dictionary
A dictionary in NLP is a structured database that contains word
definitions, pronunciations, word senses, parts of speech, and
sometimes etymology. NLP dictionaries are often used to:
• Retrieve Word Meanings: Used in WSD to distinguish between different
word senses.
• Tokenization and POS Tagging: Dictionaries help map words to their
corresponding parts of speech.
• Named Entity Recognition (NER): Dictionaries of names, places, and
organizations are used to classify entities.
Examples of NLP Dictionaries:
• WordNet
• Merriam-Webster or Oxford Dictionaries (API-based)
• Domain-Specific Dictionaries
The Lesk Algorithm
• By far the most well-studied dictionary-based algorithm for sense
disambiguation is the Lesk algorithm,
Thesaurus
A thesaurus in NLP provides lists of synonyms and antonyms, helping expand
the semantic richness of applications. Thesauri are used to:
• Enhance Text Generation: By replacing words with synonyms to avoid
repetition and improve style.
• Query Expansion in Search Engines: Expands user queries with related
words for better search results.
• Semantic Analysis and Text Summarization: Helps identify conceptually
similar words for effective summarization.
Examples of NLP Thesauri:
• Roget’s Thesaurus: A classic thesaurus of English synonyms.
• WordNet Synonym Sets (Synsets): Provides structured synonym and
antonym relationships.
• Online Thesaurus APIs: Such as Thesaurus.com API.
Applications of Dictionary and
Thesaurus in NLP
•In Word Sense Disambiguation (WSD):
Uses dictionaries (e.g., WordNet) to identify the correct meaning of
ambiguous words based on context.
Ex: "The bank raised its interest rates." (financial sense vs. riverbank).
•In Sentiment Analysis:
Thesauri help in identifying synonyms/antonyms of positive or negative
sentiment words.
Ex: Recognizing “joyful,” “elated,” and “cheerful” as positive words.
•In Text Generation and Paraphrasing:
Thesauri enhance paraphrasing by suggesting alternative wordings.
Ex: “happy” → “content,” “elated,” or “joyous.”
•Spell Checking and Auto-correction:
Bootstrapping methods
• Bootstrapping is a non-parametric method that repeatedly resamples
data from the original dataset (with replacement) to generate multiple
"bootstrap samples." These samples are used to compute estimates for
parameters like mean, variance, and accuracy.
Steps in Bootstrapping (General Process):
1.Resampling:
1. Select samples with replacement from the dataset.
2.Model Training:
1. Train the model on bootstrap samples.
3.Evaluation and Aggregation:
1. Evaluate performance metrics (e.g., accuracy) on resampled datasets and
aggregate results to reduce variance.
• Both the supervised approach and the dictionary-based approach to
WSD require large hand-built resources; supervised training sets in
one case, largedictionaries in the other.

• We can instead use bootstrapping algorithms, often called semi-

supervised learning or minimally supervised learning, which need
only a very small hand-labeled training set.

• The most widely emulated bootstrapping algorithm for WSD is the

Yarowsky algorithm (Yarowsky, 1995)
Types of Bootstrapping Methods
• Self-Training (Single-View Bootstrapping)
• Co-Training (Multi-View Bootstrapping)
• Multi-Task Bootstrapping
• Expectation-Maximization (EM) Bootstrapping
1. Self-Training
• Train a model on a small labeled dataset.
• Use the model to label unlabeled data with high-confidence predictions.
• Add the confident labels back to the training set and retrain.
• Applications: Named Entity Recognition (NER), Sentiment Analysis.
• Advantages: Simple, easy to implement.
• Challenges: Risk of error propagation.
Types of Bootstrapping Methods
2. Co-Training:
Train two classifiers on different views of the same data.
Each classifier labels some unlabeled data, which is used to retrain the
other.
Example:
Classifier 1: Uses word-based features (bag of words).
Classifier 2: Uses syntactic features (POS tags).
Advantages: Reduces bias, leverages independent feature sets.
Challenges: Requires natural feature splits in the data.
Types of Bootstrapping Methods
3.Multi-Task Bootstrapping
• Leverages shared information across multiple NLP tasks.
• Example: POS tagging and syntactic parsing.
• Helps improve performance by sharing useful learning signals.
4. Expectation-Maximization (EM) Bootstrapping
• Alternates between predicting missing labels (Expectation step) and
optimizing model parameters (Maximization step).
• Example: Estimating word probabilities in a partially labeled dataset.
Word similarity using thesaurus and Distributional
methods
• What is Word Similarity?
Measure of how closely two words are related in meaning.
Used in NLP applications like text classification, sentiment analysis, and
machine translation.

Approaches to Word Similarity:

• Thesaurus-Based Methods
• Distributional Methods
Thesaurus-Based Methods
Use predefined lexical resources, such as WordNet or Roget’s Thesaurus, to
measure word similarity.
Computes similarity based on hierarchical relationships between words
(synonyms, antonyms, hypernyms, etc.).
Example of Thesaurus-Based Word Similarity:
Using WordNet:
• Word Pairs: Car and Truck
• Similarity is calculated based on:
• Path distance in the hierarchy.
• Common parent nodes (hypernyms).
• Semantic relationships like synonymy and hyponymy.
Advantages of Thesaurus-Based Methods
• Provides human-curated lexical relationships.
• Suitable for understanding semantic word similarity (e.g., synonyms).
• Useful in ontology-based NLP applications.
Distributional Methods
• Based on the Distributional Hypothesis: Words that occur in similar
contexts tend to have similar meanings
• Measure similarity using word co-occurrence statistics in large
corpora.
• Common techniques:
• Bag of Words (BoW)
• Word Embeddings (Word2Vec, GloVe)
Distributional Word Similarity Techniques
• Bag of Words (BoW):
• Represents words based on their frequency in documents.
• Limitations: No word order or context information.
• Word Embeddings (Dense Vectors):
• Techniques: Word2Vec, GloVe, FastText.
• Represent words as dense vectors in high-dimensional space.
• Capture semantic similarity (e.g., king – queen > man – woman).
Example of Distributional Word Similarity
• Word2Vec Embedding:
• Similarity between dog and cat based on their vector proximity in embedding
space.
• Visualized as points in high-dimensional space with clustering of semantically
related words.