Let's Run Jinyeah

Bayes Decision Theory 본문

Deep Learning/Theory

Bayes Decision Theory

jinyeah 2022. 5. 15. 17:00

Prior Probability

  • Probability derived by deductive reasoning
  • calculated by existing information regarding a situation, and thus may vary depending the given situation
  • ex) 특정 fish image가 돔(S1)에 속할 확률

Class-conditional Probability / Likelihood

class-conditional PDF

  • probability density function for X(feature), given that the corresponding class is Si
  • also called Likelihood of Si with respect to X
  • ex) 돔(S1)일 때, 몸 너비(X)가 x일 확률

Posterior Probability

Probability of Si when feature x is given

the main target for minimization of classification error

ex) 몸 너비(X)가 x일 때, 돔(S1)일 확률

 

[Binary Classification] Bayes Decision Rule for Minimum Error

1. Error Probability distribution function

Error PDF

2. Decision Boundary / Decision Rule

Decision Boundary
Decision Rule

  • aim to find a decision boundary that minimizes the probability of binary classification error

3. Relation to the posterior probability

  • Bayes rule for minimum error chooses bigger posterior probability

4. Insight

  • 분류 문제에서 우리는 feature가 주어졌을 때 해당 feature가 어느 class 속하는지 구해야한다. 즉 posterior probability를 구해서 가장 높은 확률값을 가지는 class를 정답값으로 예측한다. 하지만 posterior PDF를 모델링하는 것이 까다롭기 때문에 prior probability와 likelikhood의 조합을 사용한다.

5. Extenstion to multi-class classification

 

'Deep Learning > Theory' 카테고리의 다른 글

Transfer Learning and Domain Adaptation  (0) 2022.08.10
Entropy and Cross-Entropy  (0) 2022.07.31
Normalization  (0) 2022.06.18
Objective function/Loss function  (0) 2022.05.10
Variance & Bias  (0) 2021.08.30
Comments