-
?
(
?
i
|
?
j
)
be the loss incurred for taking action
?
i
when the state
of nature is
?
j.
action
?
i
assign the
sample into any class-
Conditional
risk
R
(
?
i
|
x
)
?
< br>j
?
c
?
(
?
i
|
?
j
)
P
(
?
j
?
j
?
1
|
x
)
for i = 1,…,a
Select
the action
?
i
for
which
R(
?
i
| x)
is minimum
R is
minimum and R in this case is called the Bayes
risk = best reasonable result that can be
achieved!
?
ij
:loss
incurred for deciding
?
i
when
the true state of nature is
?
j
g
i
(x) = -
R(
?
i
| x)
max. discriminant corresponds to min.
risk
g
i
(x)
= P(
?
i
|
x)
max. discrimination
corresponds to max. posterior
g
i
(x)
?
p(x |
?
i
)
P(
?
i
)
g
i
(x) = ln
p(x |
?
i
) + ln
P(
?
i
)
问题由估计似然概率变为估计正态分布的参数问题
极大似然估计和贝叶斯估计结果接近相同，但方法概念不同
Please
present
the
basic
ideas
of
the
maximum
likelihood
estimation
method
and
Bayesian estimation method. When do
these two methods have similar results ?
请描述最大似然估计方法和贝叶斯估计方法的基本概念。
什么情况下两个方法 有
类似的结果？
I
．
Maximum-likelihood view the parameters as quantities whose values are fixed
but unknown. The best estimate of
their value is defined to be the one that
maximizes
the probability of obtaining
the samples actually observed.
II
．
Bayesian methods view the parameters as random variables having some known
prior
distribution.
Observation
of
the
samples
converts
this
to
a
posterior
density,
thereby revising our
opinion about the true values of the
parameters.
III
．
Under the condition that the number of the training samples approaches to the
infinity,
the
estimation
of
the
mean
obtained
using
Bayesian
estimation
method
is
almost identical to that obtained using
the maximum likelihood estimation method.
最小风险决策通常有一个更低的分类准确度相比于最小错误率贝叶斯决策。
然 而，
最小风险决策能够避免可能的高风险和损失。
贝叶斯参数估计方法。
Vectorize the samples.
Calculation of the mean of all training
samples.
Calculation of the
covariance matrix
Calculation of
eigenvectors and eigenvalue of the covariance
matrix. Build the feature
space.
Feature extraction of all samples.
Calculation the feature value of every
sample.
Calculation
of the test sample feature value.
Calculation of the samples of training
samples like the above step.
Find the nearest training sample as the
result.
Exercises
1.
How to use the prior and likehood to calculate the posterior ?
What is the formula ?
怎么用先验概率和似然函数计算后验概率？公式是什么？
P(
?
j
|
x) = p(x |
?
j
) .
P(
?
j
) /
p(x)
p
(
x
)
?
?
p
(
x
|
?
)
P
(
?
)
j
j
j
?
1
j
?
2
?
P
(
?
)
?
1
,
?
P
(
?
j
j< /p>
|
x
)
?
1
2.
What’s the difference in the ideas of the minimum error Bayesian decision and minimum risk
Bayesian
decision?
What’s
the
condition
that
makes
the
minimum
error
Bayesian
decision
identical to the minimum risk Bayesian
decision?
最小误差贝叶斯决策和最小风险贝叶斯决策的概念
的差别是什么？什么情况下最小误
差贝叶斯决策和最小风险贝叶斯决策是一致的（相同的
）？
答：在两类问题中，若有
?
12
?
?
22
?
?
21
?
?
11
，即所谓对称损失函数的情况，则这时最小风<
/p>
险的贝叶斯决策和最小误差的贝叶斯决策方法显然是一致的。
the
minimum
error Bayesian
decision:
to
minimize
the
classification
error
of the
Bayesian
decision.
the minimum risk Bayesian decision: to minimize the risk of the Bayesian decision.
if
R(
?
1
| x)
< R(
?
2
|
x)
action
?
1
:
“decide
?
1
” is
taken
R(
?
1
|
x) =
?
?
11
P
(
?
1
| x) +
?
12
P(
?
2
| x)
R(
?
2
|
x) =
?
?
21
P
(
?
1
| x) +
?
22
P(
?
2
| x)
3.
A person takes a lab test of nuclear radiation and the result is positive. The test returns a
correct positive result in 99% of the
cases in which the nuclear radiation is actually
present,
and a correct negative result
in 95% of the cases in which the nuclear radiation
is not present.
Furthermore,
3%
of
the
entire
population
are
radioaetively
eontaminated.
Is
this
person
eontaminated?
一人在某实验室做了一次核辐射检测，结果是阳性的。当核辐射真正存在时，检测结果
返回正确的阳性概率是
99%
；当核辐射不存在时，结果返回正确的 阴性的概率是
95%
。
而且，所有被测人群中有
3%
的人确实被辐射污染了。那么这个人被辐射污染了吗？
答：
被辐射污染概率
P
(
?
1
)<
/p>
?
0.03
未被辐射污染概率<
/p>
P
(
?
2
)
?
0.97
X
表示阳性，
X
表示阴性，则有如下结论：
P
(
X
|
?
1
)
?
0.99
，
P
(
X
|
?
2
)
?
0.95
。
则
P<
/p>
(
?
1
|
X
)
?
P
(
X
< p>|
?
1
)
P
(
?
1
)
?
P
(
X
|
?
)
P
(
?
)
i
i
i
?
1
2
?
0.99
?
0.03
?
0.38
0.99
?
0.03
?
(1
< p>?0.95)
?
0.97
P
(
?
2
|
X
)
?
1
?
P
(
?
1
|
X
)
?
0.62
根据贝叶斯决策规则有：
P
(
?
2
|
X
)
?
P
(
?
1
|
X
)
所以这个人未被辐射污染。
4.
Please
present
the
basic
ideas
of
the
maximum
likehood
estimation
method
and
Bayesian
estimation method. When
do these two methods have similar results ?
请描述最大似然估计方法和贝叶斯估计方法的基本概念。
什么情况下两个 方法有类似的
结果？
?
，
用来估计
?
所属总体分布的某个真
答：
I.
设有一个样本集
?
，
要求我们找出估计量
?
实参数
?
使得带来的贝叶斯风险最小，这就是贝叶斯估计的概念。
(
另一种说法：
把待估计的参数看成是符合某种先验概率 分布的随机变量；
对样本进
行观测的过程，就是把先验概率密度转化为后
验概率密度，这样就利用样本的信息修正
了对参数的初始估计值
)
II.
最大似然估计法的思想很简单：
在已 经得到试验结果的情况下，
我们应该寻找使这
个结果出现的可能性最大的
那个
?
作为真
?
的估计。
III.
在训练样本数目接近无穷时，
使用贝叶斯估计方法获得的平均值估计几乎和使用最
大似然估计的方法获得的
平均值一样
题外话：