济南专业网站开发公司,扬州做公司网站,网易163企业邮箱注册,兰甘肃网站建设模式识别作业
1.说明判别分类器(如logistic回归)与上述特定类别的高斯朴素贝叶斯分类器之间的关系正是logistic回归所采用的形式。
经过第2问更加普遍的推导过程#xff1a;
对应参数为#xff1a;
二次项#xff1a; v[σ112−σ1022σ112σ102,...,σD12−σD022σD12…模式识别作业
1.说明判别分类器(如logistic回归)与上述特定类别的高斯朴素贝叶斯分类器之间的关系正是logistic回归所采用的形式。
经过第2问更加普遍的推导过程
对应参数为
二次项 v[σ112−σ1022σ112σ102,...,σD12−σD022σD12σD02]v[\frac{\sigma_{11}^2-\sigma_{10}^2}{2\sigma_{11}^2\sigma_{10}^2},...,\frac{\sigma_{D1}^2-\sigma_{D0}^2}{2\sigma_{D1}^2\sigma_{D0}^2}]v[2σ112σ102σ112−σ102,...,2σD12σD02σD12−σD02]
一次项 w[σ102μ11−σ112μ10σ112σ102,...,σD02μD1−σD12μD0σD12σD02]w[\frac{\sigma_{10}^2\mu_{11}-\sigma_{11}^2\mu_{10}}{\sigma_{11}^2\sigma_{10}^2},...,\frac{\sigma_{D0}^2\mu_{D1}-\sigma_{D1}^2\mu_{D0}}{\sigma_{D1}^2\sigma_{D0}^2}]w[σ112σ102σ102μ11−σ112μ10,...,σD12σD02σD02μD1−σD12μD0]
常数项 bln(π1−π)∑lnσi0σi1∑σ112μ102−σ102μ1122σ112σ102bln(\frac{\pi}{1-\pi})\sum ln\frac{\sigma_{i0}}{\sigma{i_1}}\sum \frac{\sigma_{11}^2\mu_{10}^2-\sigma_{10}^2\mu_{11}^2}{2\sigma_{11}^2\sigma_{10}^2}bln(1−ππ)∑lnσi1σi0∑2σ112σ102σ112μ102−σ102μ112
其中
f(x)11exp(∑vixi2wixib)f(x)\frac{1}{1exp(\sum v_ix_i^2w_ix_ib)}f(x)1exp(∑vixi2wixib)1
由于σi0σi1σi\sigma_{i0} \sigma_{i1}\sigma_{i}σi0σi1σi
发现v0.v0.v0.
二次项消失一次项和常数项如下
一次项 w[μ11−μ10σ12,...,μD1−μD0σD2]w[\frac{\mu_{11}-\mu_{10}}{\sigma_{1}^2},...,\frac{\mu_{D1}-\mu_{D0}}{\sigma_{D}^2}]w[σ12μ11−μ10,...,σD2μD1−μD0]
常数项 bln(π1−π)∑μ102−μ1122σ12bln(\frac{\pi}{1-\pi})\sum \frac{\mu_{10}^2-\mu_{11}^2}{2\sigma_{1}^2}bln(1−ππ)∑2σ12μ102−μ112
f(x)11exp(wixib)f(x)\frac{1}{1exp(w_ix_ib)}f(x)1exp(wixib)1 2.当换成更普遍的高斯函数是否仍有Logistic Regression形式
生成式高斯朴素贝叶斯分类器如下
P(y1∣X)P(X∣y1)P(y1)P(X)P(X∣y1)P(y1)P(X∣y1)P(y1)P(X∣y0)P(y0)P(y1|X) \frac{P(X|y1)P(y1)}{P(X)}\frac{P(X|y1)P(y1)}{P(X|y1)P(y1)P(X|y0)P(y0)}P(y1∣X)P(X)P(X∣y1)P(y1)P(X∣y1)P(y1)P(X∣y0)P(y0)P(X∣y1)P(y1)
11P(X∣y1)P(y1)P(X∣y0)P(y0)11exp(ln(P(X∣y1)P(y1)P(X∣y0)P(y0)))\frac{1}{1\frac{P(X|y1)P(y1)}{P(X|y0)P(y0)}}\frac{1}{1exp(ln(\frac{P(X|y1)P(y1)}{P(X|y0)P(y0)}))}1P(X∣y0)P(y0)P(X∣y1)P(y1)11exp(ln(P(X∣y0)P(y0)P(X∣y1)P(y1)))1
其中
ln(P(X∣y1)P(y1)P(X∣y0)P(y0))ln(π1−π)lnP(X∣y1)P(X∣y0)ln(\frac{P(X|y1)P(y1)}{P(X|y0)P(y0)})ln(\frac{\pi}{1-\pi})ln\frac{P(X|y1)}{P(X|y0)}ln(P(X∣y0)P(y0)P(X∣y1)P(y1))ln(1−ππ)lnP(X∣y0)P(X∣y1)
ln(π1−π)ln((2πσi1)−1exp(−(X−μ1)2/2σ12)(2πσi0)−1exp(−(X−μ0)2/2σ02))ln(\frac{\pi}{1-\pi}) ln(\frac{(\sqrt{2\pi}\sigma_{i1})^{-1}exp(-(X-\mu_1)^2/2\sigma_1^2)}{(\sqrt{2\pi}\sigma_{i0})^{-1}exp(-(X-\mu_0)^2/2\sigma_0^2)})ln(1−ππ)ln((2πσi0)−1exp(−(X−μ0)2/2σ02)(2πσi1)−1exp(−(X−μ1)2/2σ12))
ln(π1−π)∑ln((2πσi1)−1exp(−(X−μi1)2/2σi12)(2πσi0)−1exp(−(X−μi0)2/2σi02))ln(\frac{\pi}{1-\pi}) \sum ln(\frac{(\sqrt{2\pi}\sigma_{i1})^{-1}exp(-(X-\mu_{i1})^2/2\sigma_{i1}^2)}{(\sqrt{2\pi}\sigma_{i0})^{-1}exp(-(X-\mu_{i0})^2/2\sigma_{i0}^2)})ln(1−ππ)∑ln((2πσi0)−1exp(−(X−μi0)2/2σi02)(2πσi1)−1exp(−(X−μi1)2/2σi12))
ln(π1−π)∑(lnσi0σi1xi2σi12−σi022σi12σi02xiσi02μi1−σi12μi0σi12σi02σi12μi02−σi02μi122σi12σi02)ln(\frac{\pi}{1-\pi}) \sum{(ln\frac{\sigma_{i0}}{\sigma{i_1}}x_i^2\frac{\sigma_{i1}^2-\sigma_{i0}^2}{2\sigma_{i1}^2\sigma_{i0}^2}x_i\frac{\sigma_{i0}^2\mu_{i1}-\sigma_{i1}^2\mu_{i0}}{\sigma_{i1}^2\sigma_{i0}^2}\frac{\sigma_{i1}^2\mu_{i0}^2-\sigma_{i0}^2\mu_{i1}^2}{2\sigma_{i1}^2\sigma_{i0}^2})}ln(1−ππ)∑(lnσi1σi0xi22σi12σi02σi12−σi02xiσi12σi02σi02μi1−σi12μi02σi12σi02σi12μi02−σi02μi12)
所以
当σi12σi02\sigma_{i1}^2\sigma_{i0}^2σi12σi02时xi2x_i^2xi2项不复存在其对应形式刚好为logistic regression。
对应参数为
二次项 v[σ112−σ1022σ112σ102,...,σD12−σD022σD12σD02]v[\frac{\sigma_{11}^2-\sigma_{10}^2}{2\sigma_{11}^2\sigma_{10}^2},...,\frac{\sigma_{D1}^2-\sigma_{D0}^2}{2\sigma_{D1}^2\sigma_{D0}^2}]v[2σ112σ102σ112−σ102,...,2σD12σD02σD12−σD02]
一次项 w[σ102μ11−σ112μ10σ112σ102,...,σD02μD1−σD12μD0σD12σD02]w[\frac{\sigma_{10}^2\mu_{11}-\sigma_{11}^2\mu_{10}}{\sigma_{11}^2\sigma_{10}^2},...,\frac{\sigma_{D0}^2\mu_{D1}-\sigma_{D1}^2\mu_{D0}}{\sigma_{D1}^2\sigma_{D0}^2}]w[σ112σ102σ102μ11−σ112μ10,...,σD12σD02σD02μD1−σD12μD0]
常数项 bln(π1−π)∑lnσi0σi1∑σ112μ102−σ102μ1122σ112σ102bln(\frac{\pi}{1-\pi})\sum ln\frac{\sigma_{i0}}{\sigma{i_1}}\sum \frac{\sigma_{11}^2\mu_{10}^2-\sigma_{10}^2\mu_{11}^2}{2\sigma_{11}^2\sigma_{10}^2}bln(1−ππ)∑lnσi1σi0∑2σ112σ102σ112μ102−σ102μ112
其中
f(x)11exp(∑vixi2wixib)f(x)\frac{1}{1exp(\sum v_ix_i^2w_ix_ib)}f(x)1exp(∑vixi2wixib)1 3.非朴素高斯贝叶斯分类器是否仍具有Logistic Regress的性质
P(y∣X)P(x1,x2∣y1)P(y1)P(X)P(x1,x2∣y1)P(y1)P(x1,x2∣y1)P(y1)P(x1,x2∣y0)P(y0)P(y|X)\frac{P(x1,x2|y1)P(y1)}{P(X)}\frac{P(x1,x2|y1)P(y1)}{P(x1,x2|y1)P(y1)P(x1,x2|y0)P(y0)}P(y∣X)P(X)P(x1,x2∣y1)P(y1)P(x1,x2∣y1)P(y1)P(x1,x2∣y0)P(y0)P(x1,x2∣y1)P(y1)
11P(x1,x2∣y1)P(y1)P(x1,x2∣y0)P(y0)11exp(e)\frac{1}{1\frac{P(x1,x2|y1)P(y1)}{P(x1,x2|y0)P(y0)}} \frac{1}{1exp(e)}1P(x1,x2∣y0)P(y0)P(x1,x2∣y1)P(y1)11exp(e)1
其中
elnπ1−πln(P(x1,x2∣y1)P(x1,x2∣y0))e ln{\frac{\pi}{1-\pi}} ln(\frac{P(x1,x2|y1)}{P(x1,x2|y0)})eln1−ππln(P(x1,x2∣y0)P(x1,x2∣y1))
由于
P(x1,x2∣yk)12πσ1σ21−p2exp(略去)P(x1,x2|yk)\frac{1}{2\pi\sigma_1\sigma_2\sqrt{1-p^2}}exp(略去)P(x1,x2∣yk)2πσ1σ21−p21exp(略去)
将其带入式子eee中得到
e∗2(1−p2)σ12σ22lnπ1−πx12(σ22−σ22)x22(σ12−σ12)x1(−2μ10σ222pσ1σ2μ202μ11σ22−2pσ1σ2μ21)x2(−2μ20σ122pσ1σ2μ102μ21σ12−2pσ1σ2μ11)x1x2(−2pσ1σ22pσ1σ2)−2pσ1σ2μ10μ202pσ1σ2μ11μ21e*2(1-p^2)\sigma_1^2\sigma_2^2 ln{\frac{\pi}{1-\pi}} x_1^2(\sigma_2^2-\sigma_2^2)x_2^2(\sigma_1^2-\sigma_1^2)x_1(-2\mu_{10}\sigma_2^22p\sigma_1\sigma_2\mu_{20}2\mu_{11}\sigma_2^2-2p\sigma_1\sigma_2\mu_{21})x_2(-2\mu_{20}\sigma_1^22p\sigma_1\sigma_2\mu_{10}2\mu_{21}\sigma_1^2-2p\sigma_1\sigma_2\mu_{11})x_1x_2(-2p\sigma_1\sigma_22p\sigma_1\sigma_2)-2p\sigma_1\sigma_2\mu_{10}\mu_{20}2p\sigma_1\sigma_2\mu_{11}\mu_{21}e∗2(1−p2)σ12σ22ln1−ππx12(σ22−σ22)x22(σ12−σ12)x1(−2μ10σ222pσ1σ2μ202μ11σ22−2pσ1σ2μ21)x2(−2μ20σ122pσ1σ2μ102μ21σ12−2pσ1σ2μ11)x1x2(−2pσ1σ22pσ1σ2)−2pσ1σ2μ10μ202pσ1σ2μ11μ21
x1(−2μ10σ222pσ1σ2μ202μ11σ22−2pσ1σ2μ21)x2(−2μ20σ122pσ1σ2μ102μ21σ12−2pσ1σ2μ11)2pσ1σ2(μ11μ21−μ10μ20)x_1(-2\mu_{10}\sigma_2^22p\sigma_1\sigma_2\mu_{20}2\mu_{11}\sigma_2^2-2p\sigma_1\sigma_2\mu_{21})x_2(-2\mu_{20}\sigma_1^22p\sigma_1\sigma_2\mu_{10}2\mu_{21}\sigma_1^2-2p\sigma_1\sigma_2\mu_{11})2p\sigma_1\sigma_2(\mu_{11}\mu_{21}-\mu_{10}\mu_{20})x1(−2μ10σ222pσ1σ2μ202μ11σ22−2pσ1σ2μ21)x2(−2μ20σ122pσ1σ2μ102μ21σ12−2pσ1σ2μ11)2pσ1σ2(μ11μ21−μ10μ20)
故
b2pσ1σ2(μ11μ21−μ10μ20)/(2(1−p2)σ12σ22)b2p\sigma_1\sigma_2(\mu_{11}\mu_{21}-\mu_{10}\mu_{20})/(2(1-p^2)\sigma_1^2\sigma_2^2)b2pσ1σ2(μ11μ21−μ10μ20)/(2(1−p2)σ12σ22)
w1(−2μ10σ222pσ1σ2μ202μ11σ22−2pσ1σ2μ21)/(2(1−p2)σ12σ22)w_1 (-2\mu_{10}\sigma_2^22p\sigma_1\sigma_2\mu_{20}2\mu_{11}\sigma_2^2-2p\sigma_1\sigma_2\mu_{21})/(2(1-p^2)\sigma_1^2\sigma_2^2)w1(−2μ10σ222pσ1σ2μ202μ11σ22−2pσ1σ2μ21)/(2(1−p2)σ12σ22)
w2(−2μ20σ122pσ1σ2μ102μ21σ12−2pσ1σ2μ11)/(2(1−p2)σ12σ22)w_2 (-2\mu_{20}\sigma_1^22p\sigma_1\sigma_2\mu_{10}2\mu_{21}\sigma_1^2-2p\sigma_1\sigma_2\mu_{11})/(2(1-p^2)\sigma_1^2\sigma_2^2)w2(−2μ20σ122pσ1σ2μ102μ21σ12−2pσ1σ2μ11)/(2(1−p2)σ12σ22)
则原式可以写成
P(y∣X)11exp(bw1x1w2x2)P(y|X)\frac{1}{1exp(bw_1x_1w_2x_2)}P(y∣X)1exp(bw1x1w2x2)1
因此仍然满足logistic regression形式。
