🧠 Soft Computing Unit 2
ANN + Fuzzy Logic!
Backpropagation, Hebb Rule, Fuzzy Sets, Membership Functions — sab important hai. Diagrams aur formulas dhyan se padhna!

🧠 Soft Computing

Unit 2 — ANN Architectures, Backpropagation, Hebb Network & Fuzzy Logic

📚 Unit 2 — Index

🏗️ Ch-2.1 — ANN Architectures & Learning
🗺️ CH-2.1 MIND MAP
SLFF → Single Layer Feedforward → Input directly connected to output → No hidden layers → Only linear problems
MLFF → Multiple hidden layers → Non-linear problems → Universal approximator → Trained by Backpropagation
McCulloch-Pitts (MCP) → First neuron model (1943) → Binary inputs/output → Threshold logic → All weights fixed (1 or 0)
Supervised Learning → Labeled data → Teacher present → Minimize error → Backpropagation, Perceptron
Unsupervised Learning → No labels → Find structure → Clustering → Hebb Rule, SOM, K-Means
Backpropagation → Forward pass (compute output) → Compute error → Backward pass (propagate error) → Update weights via gradient descent
Hebb Network → "Fire together, wire together" → Unsupervised → Weight update: Δw = α × x × y → Correlational learning
1. Single Layer Feedforward (SLFF) Architecture
📖 Single Layer Feedforward Network:
The simplest ANN architecture where input units are connected directly to output units. There are no hidden layers. Data flows in one direction only — from input to output (feedforward). Also called Single Layer Perceptron.
Single Layer Feedforward Architecture: INPUT LAYER OUTPUT LAYER (No processing) (Computes output) x₁ ────w₁₁────→ y₁ x₁ ────w₁₂────→ y₂ x₂ ────w₂₁────→ y₁ x₂ ────w₂₂────→ y₂ x₃ ────w₃₁────→ y₁ x₃ ────w₃₂────→ y₂ 3 inputs 2 outputs Each yⱼ = f(Σ wᵢⱼ × xᵢ + bⱼ) ← fully connected → every input to every output
Single Layer Feedforward Network — No hidden layers
Characteristics of SLFF:
Layers: Input layer + Output layer only (no hidden layer)
Direction: Strictly forward — no cycles, no feedback
Computation: Each output neuron: yⱼ = f(Σ wᵢⱼxᵢ + bⱼ)
Limitation: Can only solve linearly separable problems
Training: Perceptron Learning Rule or Delta Rule
Example tasks: AND gate, OR gate, simple pattern classification
✅ Advantages: Simple structure | Fast computation | Easy to understand | Works well for linearly separable data

❌ Disadvantages: Cannot solve XOR or non-linear problems | No internal representation | Limited expressive power
2. Multilayer Feedforward (MLFF) Architecture
📖 Multilayer Feedforward Network (MLFF):
An ANN architecture with one or more hidden layers between the input and output layers. Each layer is fully connected to the next. Data flows strictly forward (input → hidden(s) → output). Also called Multi-Layer Perceptron (MLP).

Key Theorem: An MLFF with even one hidden layer is a Universal Approximator — it can approximate any continuous function given enough neurons.
Multilayer Feedforward Architecture: INPUT HIDDEN HIDDEN OUTPUT LAYER LAYER 1 LAYER 2 LAYER x₁ ──→ ● ──→ ● ──→ ● ╱ ╲ ╱ ╲ ╱ ╲ x₂ ──→ ● ──→ ● ──→ ● ──→ y₁ ╲ ╱ ╲ ╱ ╲ ╱ x₃ ──→ ● ──→ ● ──→ ● ──→ y₂ Input H₁=4 H₂=3 Output (3) neurons neurons (2) Direction: LEFT ──────────────────────→ RIGHT only Each layer is FULLY CONNECTED to the next layer
Multilayer Feedforward — 2 hidden layers shown
Characteristics of MLFF:
Layers: Input + one or more Hidden + Output
Hidden neurons: Create internal representations of data
Non-linearity: Non-linear activation functions (ReLU, sigmoid, tanh) in hidden layers
Training: Backpropagation algorithm (gradient descent)
Power: Can solve ANY problem — linearly separable or not
Deep Networks: Many hidden layers = Deep Neural Network (Deep Learning)
FeatureSLFF (Single Layer)MLFF (Multi Layer)
Hidden LayersNoneOne or more
Problem TypeOnly linear/separableAny (linear + non-linear)
XOR ProblemCannot solve ✗Solved easily ✓
TrainingPerceptron ruleBackpropagation
Universal ApproximatorNoYes ✓
ComplexitySimple, fastComplex, more compute
ApplicationsAND, OR gateImage recognition, NLP, XOR
3. McCulloch-Pitts (MCP) Neuron
📖 McCulloch-Pitts Neuron (1943):
Proposed by Warren McCulloch (neurophysiologist) and Walter Pitts (mathematician). It is the first mathematical model of a neuron — a simplified, idealized version of a biological neuron. It computes a boolean function of its binary inputs.
McCulloch-Pitts Neuron: x₁ ─────────────┐ x₂ ─────────────┤ x₃ ─────────────┼──→ [Σ xᵢ] ──→ [θ threshold] ──→ output y . │ xₙ ─────────────┘ y = 1 if Σxᵢ ≥ θ (Inhibitory)──╌╌╌ y = 0 if Σxᵢ < θ Rules: • All inputs are binary (0 or 1) • Weights are fixed: excitatory = +1, inhibitory blocks firing • If ANY inhibitory input is 1 → output = 0 (absolute inhibition) • Otherwise: sum excitatory inputs, compare with threshold θ
McCulloch-Pitts Neuron Model
Properties of MCP Neuron:
1. Binary inputs: Each input xᵢ is either 0 or 1
2. Binary output: Output y is either 0 or 1 (fires or doesn't)
3. Fixed weights: Connections are either excitatory (+1) or inhibitory (absolute block)
4. Threshold logic: Neuron fires if Σ(excitatory inputs) ≥ threshold θ
5. Absolute inhibition: If any inhibitory input = 1 → output forced to 0
6. Limitations: Weights are NOT learned — manually set. Cannot handle real-valued inputs.
MCP Neuron for Logical Gates:
GateInputsθ (Threshold)Logic
ANDx₁, x₂ (excitatory)θ = 2y=1 only if x₁+x₂ ≥ 2, i.e., both =1
ORx₁, x₂ (excitatory)θ = 1y=1 if x₁+x₂ ≥ 1, i.e., at least one =1
NOTx₁ (inhibitory)θ = 1 (no excitatory)If x₁=1 → inhibited → y=0; x₁=0 → y=1
NANDx₁, x₂ excit, x₃ inhibθ = 2Combination of AND + NOT
💡 Example — MCP for AND gate (θ=2):
x₁=0, x₂=0: Σ=0, 0 < 2 → y=0 ✓
x₁=1, x₂=0: Σ=1, 1 < 2 → y=0 ✓
x₁=0, x₂=1: Σ=1, 1 < 2 → y=0 ✓
x₁=1, x₂=1: Σ=2, 2 ≥ 2 → y=1 ✓ — AND gate works!
💡 Significance of MCP Neuron:
• First mathematical proof that neurons can perform logical computations
• Showed that any boolean function can be implemented by a network of MCP neurons
• Inspired all future neural network research
Limitation: Weights are fixed (not learnable) — solved later by Hebb rule and Perceptron
4. Supervised vs Unsupervised Learning
📖 Supervised Learning:
Learning with a teacher. The network is trained on labeled data — each input has a known, correct output (label). The network adjusts weights to minimize error between its output and the correct label.

📖 Unsupervised Learning:
Learning without a teacher. No labels provided. The network discovers hidden patterns, structure, or clusters in the input data on its own.
FeatureSupervised LearningUnsupervised Learning
Training DataLabeled (input + correct output)Unlabeled (input only)
Teacher/FeedbackPresent — error signal ✓Not present ✗
GoalLearn input→output mappingDiscover structure/patterns
Error CalculationError = Target - OutputNo error (no target)
ANN MethodsBackpropagation, Perceptron, ADALINEHebb Network, SOM, K-Means
ExamplesClassification, Regression, PredictionClustering, Dimensionality reduction
ApplicationsSpam filter, image recognition, medical diagnosisCustomer segmentation, anomaly detection
DifficultyNeeds labeled data (expensive)Works with any data (more scalable)
💡 Analogy:
Supervised: Student learns with a teacher who corrects every answer — "This is a cat" (label given).
Unsupervised: Student sees many animals and groups similar ones together without being told what each is.
💡 Also important: Reinforcement Learning
Agent learns by interaction with environment. Gets reward for good actions, penalty for bad. No explicit labels but has feedback signal. Examples: Game playing (AlphaGo, Chess AI), Robot navigation.
5. Backpropagation Network (Supervised Learning)
📖 Backpropagation (BP):
The most widely used supervised learning algorithm for training Multi-Layer Feedforward Networks. Proposed by Rumelhart, Hinton & Williams (1986). It propagates the error signal backwards through the network to adjust weights using gradient descent.

Full name: Backpropagation of Error Algorithm
Key idea: Use chain rule of calculus to compute how much each weight contributed to the error.
Backpropagation — Two Phases: PHASE 1: FORWARD PASS (left to right) Input x → Hidden Layer(s) → Output y Compute activations layer by layer Calculate Error: E = ½ × Σ(target - output)² PHASE 2: BACKWARD PASS (right to left) Output Layer ↑ error signal Hidden Layer 2 ↑ error signal propagated back Hidden Layer 1 ↑ error signal propagated back Input Layer At each layer: compute ∂E/∂w (gradient) Update: w = w - α × (∂E/∂w) ↑ ↑ old weight gradient step
Backpropagation — Forward then Backward
Backpropagation Algorithm — Step by Step:

Initialization:
• Initialize all weights with small random values (e.g., between -0.5 and 0.5)
• Set learning rate α (e.g., 0.1), number of epochs

For each training example (x, target):

Step 1 — Forward Pass:
• Feed input x through network layer by layer
• Compute net input at each neuron: netⱼ = Σ wᵢⱼ × aᵢ + bⱼ
• Apply activation: aⱼ = f(netⱼ) [e.g., sigmoid]
• Get final output y = aₒᵤₜₚᵤₜ

Step 2 — Compute Error:
• E = ½ × Σ (targetₖ - yₖ)² for all output neurons k

Step 3 — Backward Pass (Output layer):
• Compute error signal (delta) for output neuron k:
   δₖ = (targetₖ - yₖ) × f'(netₖ)
   For sigmoid: f'(net) = y × (1 - y)
   So: δₖ = (targetₖ - yₖ) × yₖ × (1 - yₖ)

Step 4 — Backward Pass (Hidden layers):
• Compute delta for each hidden neuron j:
   δⱼ = f'(netⱼ) × Σₖ(δₖ × wⱼₖ)
   = aⱼ(1 - aⱼ) × Σₖ(δₖ × wⱼₖ)

Step 5 — Update Weights:
• Δwᵢⱼ = α × δⱼ × aᵢ
• wᵢⱼ(new) = wᵢⱼ(old) + Δwᵢⱼ

Step 6 — Repeat for all training examples, for multiple epochs until error converges.
Key Formulas — Backpropagation:

Error:         E = ½ Σₖ (tₖ - yₖ)²
Output delta:   δₖ = (tₖ - yₖ) · f'(netₖ) = (tₖ - yₖ) · yₖ(1-yₖ) [sigmoid]
Hidden delta:   δⱼ = f'(netⱼ) · Σₖ δₖwⱼₖ = aⱼ(1-aⱼ) · Σₖ δₖwⱼₖ
Weight update:   wᵢⱼ = wᵢⱼ + α · δⱼ · aᵢ

Where: α = learning rate | tₖ = target | yₖ = output | aᵢ = activation of previous layer
✅ Advantages of Backpropagation:
• Can train networks with any number of hidden layers
• Mathematically well-founded (gradient descent)
• General purpose — works for classification, regression
• Scales to large networks (modern deep learning uses it)
⚠️ Problems with Backpropagation:
1. Vanishing Gradient: Gradients become very small in early layers → slow/no learning in deep networks
2. Local Minima: May get stuck in local minimum of error surface instead of global minimum
3. Slow Convergence: May need many epochs to converge
4. Sensitivity to Learning Rate: Too high → oscillates, too low → very slow
5. Overfitting: Network memorizes training data, fails on new data
6. Hebb Network (Unsupervised Learning)
📖 Hebb's Rule / Hebb Network (1949):
Proposed by Donald Hebb in his book "The Organization of Behavior" (1949). Based on the biological principle:

"When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased."

Simplified: "Neurons that fire together, wire together."
If two neurons are active simultaneously, the connection between them is strengthened.
Hebb Learning Rule:

Δwᵢⱼ = α × xᵢ × yⱼ

wᵢⱼ(new) = wᵢⱼ(old) + α × xᵢ × yⱼ

Where:
α     = learning rate (positive constant)
xᵢ     = input from neuron i
yⱼ     = output of neuron j
Δwᵢⱼ = weight change between neurons i and j

Key Insight: Weight increases when both input AND output are simultaneously active (+1).
If either is 0 or negative → weight decreases or unchanged.
Hebb Learning Algorithm:
1. Initialize all weights wᵢⱼ = 0
2. For each training pattern (x):
   a. Compute output: y = f(Σ wᵢ × xᵢ)
   b. Update weights: wᵢ(new) = wᵢ(old) + α × xᵢ × y
   c. Update bias: b(new) = b(old) + α × y
3. Repeat for all patterns

Note: No error calculation! No target output needed — purely correlational.
💡 Example — Hebb Rule for AND gate (α=1, initial w₁=w₂=b=0):
Training patterns: (1,1,1)→1, (1,-1,-1)→-1, (-1,1,-1)→-1, (-1,-1,1)→1 [bipolar encoding]

Pattern 1: x₁=1, x₂=1, target=1 (use y=target in Hebb):
w₁ = 0 + 1×1×1 = 1, w₂ = 0 + 1×1×1 = 1, b = 0 + 1×1 = 1

Pattern 2: x₁=1, x₂=-1, target=-1:
w₁ = 1 + 1×1×(-1) = 0, w₂ = 1 + 1×(-1)×(-1) = 2, b = 1 + 1×(-1) = 0

(Continue for all patterns... final weights encode the AND function)
FeatureBackpropagationHebb Network
Learning TypeSupervisedUnsupervised
Teacher/LabelRequired ✓Not needed ✗
Error SignalYes — (target - output)No error computed
Weight Updatew = w + α·δ·x (error-based)w = w + α·x·y (correlation)
ArchitectureMulti-layer (hidden layers)Usually single layer
ApplicationClassification, regressionAssociative memory, pattern storage
Biological BasisLess directDirectly mirrors synaptic plasticity
💡 Hebb Network — Key Points:
• Oldest and simplest learning rule in neural networks
• Biologically plausible — mirrors how synapses strengthen in real brains
• Used as basis for associative memory networks (Hopfield networks)
• Weight grows without bound if not controlled — needs weight decay
• Does NOT minimize error — just reinforces correlations
🔮 Ch-2.2 — Introduction to Fuzzy Logic
🗺️ CH-2.2 MIND MAP
Crisp Set → Binary membership (0 or 1) → Either in or out → Sharp boundary
Fuzzy Set → Membership degree in [0,1] → Partial membership → Gradual boundary
Membership Function μ(x) → Triangular | Trapezoidal | Gaussian | Singleton
Properties → Normality | Convexity | Support | Core | α-cut | Cardinality
Operations → Union: max(μA,μB) | Intersection: min(μA,μB) | Complement: 1-μA
Fuzzy If-Then Rules → "IF x is TALL THEN y is HEAVY" | Antecedent → Consequent
Fuzzy Relations → Membership in Cartesian product A×B → Matrix representation
Fuzzy Controller → Fuzzification → Rule Evaluation → Aggregation → Defuzzification
1. Fuzzy Sets and Crisp Sets
📖 Crisp Set (Classical Set):
A traditional set where an element either belongs completely (membership = 1) or does not belong (membership = 0) — no in-between. Based on Aristotelian bivalent logic.

Characteristic Function: χ_A(x) = 1 if x ∈ A, else 0
Example: "Adults" = {people with age ≥ 18}. A 17-year-old is NOT an adult (0). An 18-year-old IS an adult (1). Sharp boundary.
📖 Fuzzy Set:
Introduced by Lotfi A. Zadeh (1965). A set where elements have degrees of membership between 0 and 1 (inclusive). Allows partial membership — reflecting the gradual nature of real-world concepts.

Membership Function: μ_A(x) ∈ [0, 1]
μ_A(x) = 1 → fully in A | μ_A(x) = 0 → not in A | 0 < μ_A(x) < 1 → partially in A

Example: "Tall people" fuzzy set: person at 5'5" might have μ = 0.3 (somewhat tall), 6'0" has μ = 0.8 (quite tall), 6'5" has μ = 1.0 (definitely tall).
Crisp Set "Adults" (age ≥ 18): Fuzzy Set "Young": Membership Membership 1 | ┌────────── 1 |──────╲ | │ | ╲ 0 |─────────────────┘ 0 | ╲──────── └────────────────────── age └──────────────── age 18 0 20 35 50 Crisp: sharp boundary at 18 Fuzzy: gradual decrease 17 → 0, 18 → 1 (abrupt jump!) 20→1, 30→0.5, 40→0, etc.
Crisp Set vs Fuzzy Set — membership comparison
FeatureCrisp SetFuzzy Set
MembershipBinary: 0 or 1 onlyContinuous: any value in [0,1]
BoundarySharp, well-definedGradual, vague
LogicTwo-valued (true/false)Multi-valued (degrees)
Real-world matchPoor for vague conceptsGood for human language
Functionχ_A(x) ∈ {0,1}μ_A(x) ∈ [0,1]
Example"Is age ≥ 18?" → YES/NO"How young is person?" → 0.7
2. Properties of Fuzzy Sets
1. Normality:
A fuzzy set A is normal if its maximum membership value is 1: max μ_A(x) = 1
(At least one element has full membership)

2. Subnormal:
A fuzzy set where max μ_A(x) < 1 (no element has full membership)

3. Support:
The crisp set of all elements with μ_A(x) > 0: Supp(A) = {x | μ_A(x) > 0}

4. Core:
The crisp set of all elements with μ_A(x) = 1: Core(A) = {x | μ_A(x) = 1}

5. α-cut (Alpha-cut):
Set of elements with membership ≥ α: Aα = {x | μ_A(x) ≥ α} — gives a crisp set
Strong α-cut: A'α = {x | μ_A(x) > α}

6. Convexity:
A fuzzy set is convex if for all x₁, x₂ and λ ∈ [0,1]:
μ_A(λx₁ + (1-λ)x₂) ≥ min(μ_A(x₁), μ_A(x₂))
(No valleys in the membership function — unimodal)

7. Cardinality:
|A| = Σ μ_A(x) for discrete sets (sum of all membership values)

8. Empty Fuzzy Set:
μ_A(x) = 0 for all x — no element has any membership

9. Universal Fuzzy Set:
μ_A(x) = 1 for all x — every element has full membership
💡 Example — Properties on Fuzzy Set A = {0.2/x₁, 0.5/x₂, 1.0/x₃, 0.7/x₄, 0/x₅}:
(Notation: μ/element)

Normal? Yes — max = 1.0 ✓
Support: {x₁, x₂, x₃, x₄} (μ > 0)
Core: {x₃} (μ = 1)
0.5-cut: {x₂, x₃, x₄} (μ ≥ 0.5)
Cardinality: |A| = 0.2 + 0.5 + 1.0 + 0.7 + 0 = 2.4
3. Membership Functions
📖 Membership Function:
A mathematical function μ_A(x): X → [0,1] that maps each element x of universe X to its degree of membership in fuzzy set A. The shape of the membership function defines how membership grade changes across the universe.
Common Membership Functions:
1. Triangular Membership Function:
Defined by 3 parameters: a (left foot), b (peak), c (right foot)

μ(x) = 0                  if x ≤ a
μ(x) = (x - a)/(b - a)     if a ≤ x ≤ b
μ(x) = (c - x)/(c - b)     if b ≤ x ≤ c
μ(x) = 0                  if x ≥ c

Shape: Triangle peak at b. Simple, widely used.
Triangular (a=2, b=5, c=8): Trapezoidal (a=1,b=3,c=6,d=8): μ μ 1 | ▲ 1 | ┌──────┐ | / \ | / \ | / \ | / \ 0 |──/─────\──── x 0 |/────────────\── x 2 5 8 1 3 6 8
Triangular and Trapezoidal Membership Functions
2. Trapezoidal Membership Function:
Defined by 4 parameters: a, b, c, d (a ≤ b ≤ c ≤ d)

μ(x) = 0                   if x ≤ a
μ(x) = (x-a)/(b-a)        if a ≤ x ≤ b
μ(x) = 1                   if b ≤ x ≤ c (flat top)
μ(x) = (d-x)/(d-c)        if c ≤ x ≤ d
μ(x) = 0                   if x ≥ d

Note: When b=c (flat top disappears) → becomes Triangular!
3. Gaussian Membership Function:
μ(x) = e^(-(x-c)²/2σ²)

Parameters: c = center (peak), σ = spread (width)
Shape: Smooth bell curve. Most natural-looking. Used in Radial Basis Networks.

4. Singleton Membership Function:
μ(x) = 1 at exactly one point x₀, μ(x) = 0 elsewhere
Used in crisp output of Mamdani fuzzy controller.

5. S-Shaped (Sigmoidal):
μ(x) = 1/(1+e^(-a(x-c))) — gradual increase from 0 to 1
Used for "large", "high" linguistic variables.
FunctionParametersShapeBest Used For
Triangulara, b, cTriangle (pointed peak)Simple, symmetric linguistic terms
Trapezoidala, b, c, dTrapezoid (flat top)When range of values is "definitely in" set
Gaussianc (center), σ (spread)Bell curveNatural, smooth gradients
Singletonx₀ (single point)Spike at one pointCrisp output defuzzification
Sigmoidala, cS-curve"Large", "high", "old" — one-sided
4. Operations on Fuzzy Sets
📖 Fuzzy Set Operations:
Three basic operations are defined for fuzzy sets, analogous to classical set operations but extended to handle membership degrees in [0,1].
Three Basic Fuzzy Operations:

1. UNION (A ∪ B):    μ_(A∪B)(x) = max[μ_A(x), μ_B(x)]    (OR operation)
2. INTERSECTION (A ∩ B): μ_(A∩B)(x) = min[μ_A(x), μ_B(x)]    (AND operation)
3. COMPLEMENT (Ā):      μ_Ā(x) = 1 - μ_A(x)              (NOT operation)
Solved Numerical:
Q: Given fuzzy sets A and B on universe X = {1,2,3,4,5}:
A = {0.1/1, 0.4/2, 0.8/3, 1.0/4, 0.6/5}
B = {0.5/1, 0.7/2, 0.3/3, 0.2/4, 0.9/5}
Find: (i) A∪B   (ii) A∩B   (iii) Ā   (iv) B̄
(i) A ∪ B = max(μ_A, μ_B):
x=1: max(0.1, 0.5) = 0.5
x=2: max(0.4, 0.7) = 0.7
x=3: max(0.8, 0.3) = 0.8
x=4: max(1.0, 0.2) = 1.0
x=5: max(0.6, 0.9) = 0.9
A∪B = {0.5/1, 0.7/2, 0.8/3, 1.0/4, 0.9/5}

(ii) A ∩ B = min(μ_A, μ_B):
x=1: min(0.1, 0.5) = 0.1
x=2: min(0.4, 0.7) = 0.4
x=3: min(0.8, 0.3) = 0.3
x=4: min(1.0, 0.2) = 0.2
x=5: min(0.6, 0.9) = 0.6
A∩B = {0.1/1, 0.4/2, 0.3/3, 0.2/4, 0.6/5}

(iii) Complement Ā = 1 - μ_A:
Ā = {0.9/1, 0.6/2, 0.2/3, 0.0/4, 0.4/5}

(iv) Complement B̄ = 1 - μ_B:
B̄ = {0.5/1, 0.3/2, 0.7/3, 0.8/4, 0.1/5}
💡 De Morgan's Laws hold for Fuzzy Sets:
• Complement of Union: (A∪B)' = A' ∩ B'
• Complement of Intersection: (A∩B)' = A' ∪ B'

Important: Law of Excluded Middle DOES NOT hold for Fuzzy Sets!
In crisp sets: A ∪ Ā = X (full universe) and A ∩ Ā = ∅
In fuzzy sets: A ∪ Ā ≠ X in general (may not cover everything with full membership)
5. Fuzzy If-Then Rules
📖 Fuzzy If-Then Rules:
The knowledge representation mechanism in fuzzy systems. Rules capture human expert knowledge in linguistic form:

IF <antecedent/condition> THEN <consequent/conclusion>

Both antecedent and consequent use linguistic variables (variables whose values are words like "tall", "hot", "fast")
💡 Examples of Fuzzy If-Then Rules:

Rule 1: IF temperature is HOT THEN fan_speed is HIGH
Rule 2: IF speed is LOW AND distance is FAR THEN braking is GENTLE
Rule 3: IF age is YOUNG AND income is HIGH THEN credit_risk is LOW
Rule 4: IF temperature is COLD OR temperature is COOL THEN heater is ON
Structure of Fuzzy Rule:
Antecedent (IF part):
• Single condition: "IF x is A"
• Multiple conditions: "IF x is A AND y is B" or "IF x is A OR y is B"
• Computed using: AND → min(μ_A, μ_B), OR → max(μ_A, μ_B), NOT → 1 - μ_A

Consequent (THEN part):
• "THEN z is C" — the output linguistic variable and its term

Firing Strength (Rule activation level):
• α = degree to which antecedent is satisfied = min(μ inputs)
• The consequent membership is clipped/scaled by α
💡 Linguistic Variables:
A linguistic variable is characterized by: (Name, Term set, Universe, Membership functions)
Example: Temperature = (Name="Temperature", Terms={COLD, COOL, WARM, HOT}, Universe=[0,100°C])

Each term (COLD, COOL, WARM, HOT) has its own membership function defining its meaning.
6. Fuzzy Relations
📖 Fuzzy Relation:
A fuzzy relation R between sets X and Y is a fuzzy set defined on the Cartesian product X×Y. It assigns a membership degree μ_R(x,y) ∈ [0,1] to each pair (x,y), indicating the degree of relationship between x and y.

In crisp relations: element pairs either are (1) or are not (0) related.
In fuzzy relations: element pairs have degrees of relationship.
Fuzzy Relation Notation:

R: X × Y → [0,1]
R = {((x,y), μ_R(x,y)) | (x,y) ∈ X×Y}

Matrix Representation:
Rows = elements of X, Columns = elements of Y
Entry R[i][j] = μ_R(xᵢ, yⱼ) = degree of relation between xᵢ and yⱼ
Composition of Fuzzy Relations:
Max-Min Composition (R ∘ S):
If R ⊆ X×Y and S ⊆ Y×Z, then R∘S ⊆ X×Z:
μ_(R∘S)(x,z) = max_y [min(μ_R(x,y), μ_S(y,z))]

Meaning: To find how strongly x relates to z through the intermediate set Y:
For each intermediate y, find min of R(x,y) and S(y,z), then take the maximum over all y.
Solved Numerical — Fuzzy Relation Composition:
Q: R ⊆ X×Y and S ⊆ Y×Z given as matrices. Find R ∘ S using Max-Min composition.
R = [[0.3, 0.5], [0.8, 0.4]]
S = [[0.6, 0.2], [0.7, 0.9]]
R∘S matrix is 2×2. Each entry (i,j) = max_k[min(R[i][k], S[k][j])]

Entry (1,1):
k=1: min(R[1][1], S[1][1]) = min(0.3, 0.6) = 0.3
k=2: min(R[1][2], S[2][1]) = min(0.5, 0.7) = 0.5
max(0.3, 0.5) = 0.5

Entry (1,2):
k=1: min(0.3, 0.2) = 0.2
k=2: min(0.5, 0.9) = 0.5
max(0.2, 0.5) = 0.5

Entry (2,1):
k=1: min(0.8, 0.6) = 0.6
k=2: min(0.4, 0.7) = 0.4
max(0.6, 0.4) = 0.6

Entry (2,2):
k=1: min(0.8, 0.2) = 0.2
k=2: min(0.4, 0.9) = 0.4
max(0.2, 0.4) = 0.4

R ∘ S = [[0.5, 0.5], [0.6, 0.4]]
7. Fuzzy Controller
📖 Fuzzy Controller (Fuzzy Logic Controller - FLC):
A control system that uses fuzzy logic to map crisp input values (from sensors) to crisp output control signals (to actuators), using fuzzy rules that encode expert knowledge. Introduced by E.H. Mamdani (1974).

Applications: Air conditioners, washing machines, elevator control, camera autofocus, ABS brakes, subway systems.
Fuzzy Controller Architecture: Rule Base (IF-THEN rules) Knowledge Base │ Crisp Input ──→ [Fuzzification] ──→ [Inference Engine] ──→ [Defuzzification] ──→ Crisp Output (sensor data) (crisp→fuzzy) (apply rules) (fuzzy→crisp) (control signal) e.g., 35°C "Warm=0.6" compute output e.g., COG e.g., 65% "Hot=0.2" fuzzy set method fan speed
4 Stages of a Fuzzy Logic Controller
4 Stages of a Fuzzy Controller:

Stage 1 — Fuzzification:
Convert crisp input value(s) to fuzzy membership degrees using input membership functions.
Example: Temperature = 35°C → μ_WARM(35) = 0.6, μ_HOT(35) = 0.2

Stage 2 — Rule Evaluation (Inference):
Apply all fuzzy IF-THEN rules to the fuzzified inputs. Compute the firing strength of each rule:
• Rule 1: IF temp is WARM → FAN is MEDIUM. Firing strength α₁ = 0.6
• Rule 2: IF temp is HOT → FAN is HIGH. Firing strength α₂ = 0.2
For each rule, clip/scale the output membership function by its firing strength.

Stage 3 — Aggregation:
Combine all the clipped/scaled output fuzzy sets from all rules into one combined output fuzzy set.
Using: Union (max) of all rule outputs → one aggregate output fuzzy set

Stage 4 — Defuzzification:
Convert the aggregate output fuzzy set back to a single crisp control value.
Methods:
Centroid of Area (COG): x* = Σ[μ(x)·x] / Σ[μ(x)] — most common
Mean of Maxima (MOM): average x values where μ is maximum
Largest of Maxima (LOM): rightmost x where μ is maximum
Smallest of Maxima (SOM): leftmost x where μ is maximum
Defuzzification — Centroid Method Example:
Q: After aggregation, output fuzzy set has memberships:
FAN_SPEED: {0.6/40%, 0.6/50%, 0.2/70%, 0.2/80%}
Find crisp output using Centroid method.
Centroid (COG) = Σ[μ(x)·x] / Σ[μ(x)]

Numerator = (0.6×40) + (0.6×50) + (0.2×70) + (0.2×80)
= 24 + 30 + 14 + 16 = 84

Denominator = 0.6 + 0.6 + 0.2 + 0.2 = 1.6

Crisp output = 84/1.6 = 52.5%
→ Fan speed set to 52.5%
✅ Advantages of Fuzzy Controller:
• No need for mathematical model of the system — uses human expert rules
• Robust to noise and imprecise measurements
• Easy to understand and modify (linguistic rules)
• Handles non-linear systems naturally
• Works well when exact mathematical model is unknown
💡 Types of Fuzzy Controllers:
Mamdani FLC: Both antecedent and consequent are fuzzy sets. Output is defuzzified. Most common type.
Sugeno FLC (TSK): Consequent is a crisp function of inputs (not fuzzy set). Faster computation, easier optimization.
Ready for Exam? Sab padh liya? Ab Quick Revision karo — formulas, rules, operations aur exam tips ek jagah! Quick Revision Karo →
⚡ Quick Revision — Unit 2 Last Minute Notes
📌 How to Use: Read this 10 minutes before exam. All key formulas, comparisons and definitions — condensed and exam-ready!

🏗️ Ch-2.1 — ANN Architectures & Learning

SLFF vs MLFF:
SLFF: No hidden layers, only linear problems, Perceptron rule, cannot solve XOR
MLFF: 1+ hidden layers, any problem (non-linear too), Backpropagation, Universal Approximator
McCulloch-Pitts Neuron (1943):
• Binary inputs (0/1), Binary output (0/1)
• Fixed weights (excitatory=+1, inhibitory=blocks firing)
• Fires if: Σ(excitatory inputs) ≥ θ AND no inhibitory input active
• AND gate: θ=2 | OR gate: θ=1 | NOT gate: inhibitory input
FeatureSupervisedUnsupervised
Labels neededYes ✓No ✗
Error signalYes — (target-output)No
ANN examplesBackpropagation, PerceptronHebb Rule, SOM
ApplicationsClassification, regressionClustering, compression
Backpropagation — Key Formulas:

Error: E = ½Σ(tₖ - yₖ)²
Output delta: δₖ = (tₖ - yₖ) · yₖ(1-yₖ)   [sigmoid]
Hidden delta: δⱼ = aⱼ(1-aⱼ) · Σₖ δₖwⱼₖ
Weight update: wᵢⱼ = wᵢⱼ + α · δⱼ · aᵢ

Problems: Vanishing gradient | Local minima | Slow | Overfitting
Hebb Learning Rule:

Δwᵢⱼ = α × xᵢ × yⱼ
wᵢⱼ(new) = wᵢⱼ(old) + α × xᵢ × yⱼ

"Fire together → Wire together" | Unsupervised | No error | Correlational
Biologically plausible | Associative memory

🔮 Ch-2.2 — Fuzzy Logic

FeatureCrisp SetFuzzy Set
MembershipBinary: {0,1}Continuous: [0,1]
BoundarySharpGradual
LogicTwo-valuedMulti-valued
Functionχ(x) ∈ {0,1}μ(x) ∈ [0,1]
Fuzzy Set Properties — Quick Ref:
Normal: max μ_A(x) = 1
Support: {x | μ_A(x) > 0} — elements with ANY membership
Core: {x | μ_A(x) = 1} — elements with FULL membership
α-cut: {x | μ_A(x) ≥ α} — crisp set at threshold α
Cardinality: |A| = Σ μ_A(x)
Fuzzy Operations — MOST IMPORTANT:

Union (A ∪ B):     μ(x) = max[μ_A(x), μ_B(x)]    (OR)
Intersection (A∩B): μ(x) = min[μ_A(x), μ_B(x)]    (AND)
Complement (Ā):      μ(x) = 1 - μ_A(x)           (NOT)

De Morgan's Laws hold in Fuzzy:
(A∪B)' = A'∩B'  |  (A∩B)' = A'∪B'
⚠️ But: A ∪ Ā ≠ Universe (Law of excluded middle fails!)
Membership Functions — Quick Summary:
Triangular (a,b,c): linearly rises to peak at b, falls — simplest
Trapezoidal (a,b,c,d): rises, flat top b-c, falls — when range is "definitely" in set
Gaussian (c,σ): bell curve at center c — natural, smooth
Singleton: spike at one point — crisp output in defuzzification
Sigmoidal: S-curve — one-sided, for "large", "old"
Fuzzy Relation Composition (Max-Min):

μ_(R∘S)(x,z) = max_y [min(μ_R(x,y), μ_S(y,z))]

Step: For each (x,z) pair, compute min across each intermediate y, then take max
Fuzzy Controller — 4 Stages (EXAM IMPORTANT!):
1. Fuzzification: Crisp input → fuzzy membership degrees (using MFs)
2. Rule Evaluation: Apply IF-THEN rules, compute firing strength α for each rule
3. Aggregation: Combine all rule outputs using Union (max)
4. Defuzzification: Aggregate fuzzy set → crisp output value

Centroid method (most common):
x* = Σ[μ(x)·x] / Σ[μ(x)]
⚠️ Common Exam Mistakes:
❌ Using AND for union (use max) and OR for intersection (use min) — it's OPPOSITE of intuition!
❌ Saying fuzzy sets satisfy law of excluded middle (A ∪ Ā = X) — they DON'T
❌ Confusing Support ({x | μ>0}) with Core ({x | μ=1})
❌ Using addition for MCP neuron weights — they are 0/1 (fixed), NOT learned
❌ Forgetting that Backpropagation has TWO phases (forward + backward)
❌ Writing Hebb rule with error signal — Hebb has NO error, it's purely correlational
🌟 All the Best Bhai!
Backpropagation ke 4 steps, Hebb rule ka formula, Fuzzy operations (max/min/1-μ), aur Fuzzy Controller ke 4 stages — yeh sab guaranteed questions hain! Numericals practice karo aur tu set hai! 💪🧠
❓ Important Questions — Unit 2 (2M & 5M)
📌 10 questions of 2 marks + 5 questions of 5 marks. Both chapters covered. Click to expand answers.

📝 2 Marks Questions

2M
Q1. Compare Single Layer Feedforward and Multi Layer Feedforward architectures.
SLFFMLFF
No hidden layersOne or more hidden layers
Solves only linear problemsSolves any problem (linear + non-linear)
Cannot solve XOR ✗Solves XOR easily ✓
Perceptron learning ruleBackpropagation algorithm
Not Universal ApproximatorUniversal Approximator ✓
2M
Q2. Explain the McCulloch-Pitts neuron model with an example.
McCulloch-Pitts Neuron (1943): First mathematical model of a neuron. It has:
• Binary inputs (0 or 1), Fixed weights (excitatory +1 or inhibitory)
• Threshold θ — neuron fires (y=1) if Σ(excitatory inputs) ≥ θ
• Absolute inhibition — if any inhibitory input = 1 → y = 0

Example — AND gate (θ=2):
x₁=1, x₂=1: sum=2 ≥ 2 → y=1 ✓
x₁=1, x₂=0: sum=1 < 2 → y=0 ✓
x₁=0, x₂=0: sum=0 < 2 → y=0 ✓

Limitation: Weights are NOT learnable — must be set manually.
2M
Q3. Differentiate Supervised and Unsupervised Learning with examples.
Supervised Learning: Trains with labeled data (input + correct output). Has a teacher providing error feedback. Network minimizes error = target - output.
Methods: Backpropagation, Perceptron
Examples: Spam detection, image classification, medical diagnosis

Unsupervised Learning: Trains without labels. No teacher. Discovers hidden structure/patterns in data on its own.
Methods: Hebb Network, SOM, K-Means clustering
Examples: Customer segmentation, anomaly detection, topic modeling
2M
Q4. State Hebb's Learning Rule. What is its biological significance?
Hebb's Learning Rule (1949):
"Neurons that fire together, wire together."

Formula: wᵢⱼ(new) = wᵢⱼ(old) + α × xᵢ × yⱼ
Where α = learning rate, xᵢ = input, yⱼ = output

Key: Weight increases when BOTH input and output are simultaneously active. No error signal needed — purely correlational.

Biological Significance:
Mirrors synaptic plasticity — when two neurons fire simultaneously, the synapse between them strengthens. This is how real brains learn associations and form memories. Basis of Long-Term Potentiation (LTP) in neuroscience.
2M
Q5. What are the problems of Backpropagation algorithm?
Problems with Backpropagation:

1. Vanishing Gradient: Gradients become extremely small as they propagate back through many layers → earlier layers learn very slowly or not at all

2. Local Minima: Gradient descent may get trapped in a local minimum of error surface → not finding global optimal weights

3. Slow Convergence: May need thousands of epochs to converge, especially for complex problems

4. Sensitivity to Learning Rate: Too high → oscillates/diverges; too low → extremely slow

5. Overfitting: Network memorizes training data but fails to generalize to new data

6. Requires large labeled dataset for good performance
2M
Q6. What is a Fuzzy Set? How is it different from a Crisp Set?
Fuzzy Set (Zadeh, 1965): A set where each element has a degree of membership μ_A(x) ∈ [0,1]. Allows partial membership — an element can be "partially in" the set.

Crisp Set: Traditional set where membership is binary — either fully in (1) or fully out (0). No partial membership.

Key Difference:
Crisp: "Is 17 an adult? NO" (sharp boundary at 18)
Fuzzy: "How young is 25?" μ_YOUNG(25) = 0.7 (gradual boundary)

Crisp uses χ_A(x) ∈ {0,1}. Fuzzy uses μ_A(x) ∈ [0,1].
Fuzzy sets model real-world vagueness and linguistic concepts naturally.
2M
Q7. Define: Support, Core, and α-cut of a fuzzy set.
Support: The crisp set of all elements that have ANY membership (μ > 0).
Supp(A) = {x ∈ X | μ_A(x) > 0}
Example: A = {0.2/1, 0.5/2, 0/3} → Support = {1, 2}

Core: The crisp set of all elements with FULL membership (μ = 1).
Core(A) = {x ∈ X | μ_A(x) = 1}
Example: A = {0.5/1, 1.0/2, 0.7/3} → Core = {2}

α-cut (Alpha-cut): Crisp set of all elements with membership ≥ α.
Aα = {x ∈ X | μ_A(x) ≥ α}
Example: For A = {0.3/a, 0.7/b, 1.0/c}, the 0.5-cut = {b, c}
2M
Q8. Write the formulas for Union, Intersection and Complement of fuzzy sets with examples.
Three Basic Fuzzy Operations:

1. Union (A ∪ B) — OR: μ_(A∪B)(x) = max[μ_A(x), μ_B(x)]
2. Intersection (A ∩ B) — AND: μ_(A∩B)(x) = min[μ_A(x), μ_B(x)]
3. Complement (Ā) — NOT: μ_Ā(x) = 1 - μ_A(x)

Example: μ_A(x) = 0.3, μ_B(x) = 0.7 for some x:
Union: max(0.3, 0.7) = 0.7
Intersection: min(0.3, 0.7) = 0.3
Complement of A: 1 - 0.3 = 0.7

De Morgan's Laws hold: (A∪B)' = A'∩B' | (A∩B)' = A'∪B'
2M
Q9. What are Fuzzy If-Then rules? Give two examples.
Fuzzy If-Then Rules: Knowledge representation in fuzzy systems using linguistic variables. Structure: IF <antecedent> THEN <consequent>

Both antecedent (IF part) and consequent (THEN part) use fuzzy linguistic terms like "TALL", "HOT", "HIGH".

Firing Strength α = degree to which antecedent is satisfied:
• Single condition: α = μ_A(x)
• AND: α = min(μ_A(x), μ_B(y))
• OR: α = max(μ_A(x), μ_B(y))

Examples:
1. IF temperature is HOT THEN fan_speed is HIGH
2. IF speed is LOW AND distance is FAR THEN braking is GENTLE
3. IF pressure is HIGH OR temperature is HIGH THEN alarm is ON
2M
Q10. What are the 4 stages of a Fuzzy Logic Controller?
4 Stages of Fuzzy Logic Controller (FLC):

1. Fuzzification: Convert crisp input (e.g., sensor reading 35°C) to fuzzy membership degrees using input membership functions. Result: μ_WARM(35°C)=0.6, μ_HOT(35°C)=0.2

2. Rule Evaluation (Inference): Apply all fuzzy IF-THEN rules. For each rule, compute firing strength α using min/max. Clip or scale output membership functions by α.

3. Aggregation: Combine all clipped output fuzzy sets from all rules into one aggregate output fuzzy set using Union (max).

4. Defuzzification: Convert aggregate fuzzy output to a single crisp control value.
Common method: Centroid (COG): x* = Σ[μ(x)·x] / Σ[μ(x)]

📝 5 Marks Questions

5M
Q1. Explain the Backpropagation algorithm in detail with all formulas and steps. What are its limitations?
Backpropagation (Rumelhart, Hinton & Williams, 1986):
Supervised learning algorithm for MLFF networks. Uses gradient descent to minimize error.

2 Phases:

Phase 1 — Forward Pass:
Input x → compute activations layer by layer
netⱼ = Σ wᵢⱼ·aᵢ + bⱼ  |  aⱼ = f(netⱼ) [sigmoid: 1/(1+e⁻ˣ)]
Error: E = ½ Σₖ(tₖ - yₖ)²

Phase 2 — Backward Pass:
Output delta: δₖ = (tₖ - yₖ) · yₖ(1-yₖ)
Hidden delta: δⱼ = aⱼ(1-aⱼ) · Σₖ δₖwⱼₖ
Update: wᵢⱼ = wᵢⱼ + α · δⱼ · aᵢ

Algorithm:
1. Random initialization of weights
2. For each training example: Forward pass → Compute error → Backward pass → Update weights
3. Repeat until error converges

Limitations:
• Vanishing gradient — gradients shrink in early layers
• Local minima — may not find global optimum
• Slow convergence — needs many epochs
• Sensitive to learning rate
• Overfitting on small datasets
5M
Q2. Given A = {0.2/1, 0.6/2, 1.0/3, 0.5/4} and B = {0.4/1, 0.9/2, 0.3/3, 0.8/4}. Find: (i) A∪B (ii) A∩B (iii) Ā (iv) B̄ (v) Cardinality of A and B.
(i) A ∪ B = max(μ_A, μ_B):
x=1: max(0.2, 0.4) = 0.4
x=2: max(0.6, 0.9) = 0.9
x=3: max(1.0, 0.3) = 1.0
x=4: max(0.5, 0.8) = 0.8
A∪B = {0.4/1, 0.9/2, 1.0/3, 0.8/4}

(ii) A ∩ B = min(μ_A, μ_B):
x=1: min(0.2, 0.4) = 0.2
x=2: min(0.6, 0.9) = 0.6
x=3: min(1.0, 0.3) = 0.3
x=4: min(0.5, 0.8) = 0.5
A∩B = {0.2/1, 0.6/2, 0.3/3, 0.5/4}

(iii) Ā = 1 - μ_A:
Ā = {0.8/1, 0.4/2, 0.0/3, 0.5/4}

(iv) B̄ = 1 - μ_B:
B̄ = {0.6/1, 0.1/2, 0.7/3, 0.2/4}

(v) Cardinalities:
|A| = 0.2 + 0.6 + 1.0 + 0.5 = 2.3
|B| = 0.4 + 0.9 + 0.3 + 0.8 = 2.4
5M
Q3. Explain Fuzzy Logic Controller with its 4 stages and diagram. Give an example of defuzzification using centroid method.
Fuzzy Logic Controller (Mamdani, 1974):
Maps crisp inputs to crisp control outputs using fuzzy rules without needing mathematical model.

Architecture:
Crisp Input → [Fuzzification] → [Inference Engine] → [Aggregation] → [Defuzzification] → Crisp Output
                               ↑ Rule Base (IF-THEN rules)

4 Stages:
1. Fuzzification: temp=35°C → μ_WARM=0.6, μ_HOT=0.2
2. Inference: Rule1: IF WARM → MEDIUM FAN (α=0.6); Rule2: IF HOT → HIGH FAN (α=0.2)
3. Aggregation: Clip MEDIUM at 0.6, HIGH at 0.2; combine by max
4. Defuzzification: Find centroid of combined output fuzzy set

Centroid Example:
Combined output: {0.6/50, 0.6/60, 0.2/80, 0.2/90}
Centroid = (0.6×50 + 0.6×60 + 0.2×80 + 0.2×90) / (0.6+0.6+0.2+0.2)
= (30+36+16+18) / 1.6 = 100/1.6 = 62.5% fan speed
5M
Q4. Explain Membership Functions in detail. Describe Triangular and Trapezoidal membership functions with formulas and diagrams.
Membership Function: Mathematical function μ_A(x): X→[0,1] mapping each element to its degree of membership. Defines how we interpret linguistic terms like "TALL", "HOT", "YOUNG".

Triangular MF — Triangle(a, b, c):
μ(x) = 0              x ≤ a
μ(x) = (x-a)/(b-a)   a ≤ x ≤ b (rising)
μ(x) = (c-x)/(c-b)   b ≤ x ≤ c (falling)
μ(x) = 0              x ≥ c
Peak at b (μ=1). Used for symmetric, simple linguistic terms.

Trapezoidal MF — Trapezoid(a, b, c, d):
μ(x) = 0                x ≤ a
μ(x) = (x-a)/(b-a)    a ≤ x ≤ b (rising)
μ(x) = 1                b ≤ x ≤ c (flat top = full membership)
μ(x) = (d-x)/(d-c)    c ≤ x ≤ d (falling)
μ(x) = 0                x ≥ d
Used when a range of values are "definitely" in the set.

Other types: Gaussian (bell curve, smooth), Singleton (spike), Sigmoidal (S-curve, one-sided)
5M
Q5. Compute Max-Min composition of fuzzy relations R and S where:
R = [[0.7, 0.5], [0.3, 0.8]] and S = [[0.4, 0.9], [0.6, 0.2]]
Max-Min Composition R ∘ S:
Formula: (R∘S)[i][j] = max_k [min(R[i][k], S[k][j])]

R is 2×2, S is 2×2, so R∘S is 2×2

Entry (1,1) — row 1 of R, col 1 of S:
k=1: min(R[1][1], S[1][1]) = min(0.7, 0.4) = 0.4
k=2: min(R[1][2], S[2][1]) = min(0.5, 0.6) = 0.5
max(0.4, 0.5) = 0.5

Entry (1,2) — row 1 of R, col 2 of S:
k=1: min(R[1][1], S[1][2]) = min(0.7, 0.9) = 0.7
k=2: min(R[1][2], S[2][2]) = min(0.5, 0.2) = 0.2
max(0.7, 0.2) = 0.7

Entry (2,1) — row 2 of R, col 1 of S:
k=1: min(R[2][1], S[1][1]) = min(0.3, 0.4) = 0.3
k=2: min(R[2][2], S[2][1]) = min(0.8, 0.6) = 0.6
max(0.3, 0.6) = 0.6

Entry (2,2) — row 2 of R, col 2 of S:
k=1: min(R[2][1], S[1][2]) = min(0.3, 0.9) = 0.3
k=2: min(R[2][2], S[2][2]) = min(0.8, 0.2) = 0.2
max(0.3, 0.2) = 0.3

R ∘ S = [[0.5, 0.7], [0.6, 0.3]]