邯郸网站建设渠道,动画网页制作网站,2345网址导航 中国最,自己做店招的网站语义检索的效果确实不错#xff0c;特别是在非关键词命中的情况下。就像我这个案例。通过做数据的文本嵌入#xff0c;然后用向量的做召回。实现了语义的搜索。虽然我搜索的是“中国的首都”#xff0c;但是把命中北京的也给我召回了#xff0c;大大提升了召回效果。 其中我… 语义检索的效果确实不错特别是在非关键词命中的情况下。就像我这个案例。通过做数据的文本嵌入然后用向量的做召回。实现了语义的搜索。虽然我搜索的是“中国的首都”但是把命中北京的也给我召回了大大提升了召回效果。 其中我用全文模糊匹配。召回top10命中的数据完全不相关。但是用语义搜索召回10条数据有8条都是在描述北京。这大概就是语义检索的魅力。 普通搜索搜索 问题中国的首都 POST my_index/_search
{query: {match: {title: 中国的首都}},_source: [title]
}普通搜索的召回 {took: 11,timed_out: false,_shards: {total: 2,successful: 2,skipped: 0,failed: 0},hits: {total: {value: 52,relation: eq},max_score: 10.82337,hits: [{_index: my_index,_id: 866c3fe0033efed6047ffcc5d0e9eaff,_score: 10.82337,_source: {title: 中国的男人们在哪}},{_index: my_index,_id: 9f0927938ebe9002091ac6b319a7ba2cz1,_score: 10.301819,_source: {title: 英媒投资中国的时机“就是当下”}},{_index: my_index,_id: 1d81a61adc99565bec4ef153a6a7d236z1,_score: 9.914771,_source: {title: 研发出属于中国的“黑色黄金”}},{_index: my_index,_id: 9121f2d9959e9063db7515f3f479571bz1,_score: 9.914771,_source: {title: 研发出属于中国的“黑色黄金”}},{_index: my_index,_id: d36e2da9f11fb764fee66616139e8624,_score: 9.718359,_source: {title: 中国的国鸟、国树、国宝、国花是哪些}},{_index: my_index,_id: 8145d4e100ec51107620d383370e7bae,_score: 9.718359,_source: {title: 中国的社会公德有哪些 社会公德有哪些}},{_index: my_index,_id: 11308f9cb1a6f86ade85d9a5e3991db6,_score: 9.710967,_source: {title: 中国的崛起之路又燃又催泪}},{_index: my_index,_id: 32b77f0c98d224893a9a592c37b3fb28,_score: 9.515373,_source: {title: 特朗普提议大幅提高中国的关税}},{_index: my_index,_id: a607f403f08233c0eb397242acb438d6z1,_score: 8.973172,_source: {title: 时光穿越50年画家笔下中国的璀璨岁月}},{_index: my_index,_id: c51fac40c11bd3fa6aa95ff574444304,_score: 8.973172,_source: {title: #外国人请走开 #外国人在中国的生活}}]}
}使用模型做文本嵌入 该演示模型为m3e。文本嵌入模型。 这里是利用es的机器学习模块并用eland将模型导入到ES。我已经导入过模型了在我的其它文章中已经写过。这里只是用模型做文本嵌入。 POST _ml/trained_models/sentence-transformers__m3e-base/_infer
{docs : [{text_field: 中国的首都}]
}向量搜索 使用上一个步骤将问题转向量。然后做搜索 POST my_index/_search
{knn: {field: title_embedding.predicted_value,query_vector: [-0.19025953114032745,-0.14106161892414093,0.6414821743965149,-0.9126410484313965,0.6831074357032776,0.6467575430870056,0.1112072542309761,1.208135724067688,-0.9754570722579956,0.7777673006057739,1.0870678424835205,0.2299879938364029,0.708989143371582,-0.5606290698051453,-0.43192058801651,-0.18655601143836975,0.8479806780815125,0.4816007614135742,-1.1378719806671143,-0.980989396572113,0.568405270576477,0.11579449474811554,-1.585147738456726,0.19661249220371246,0.12738735973834991,0.627743661403656,0.1393081396818161,-0.07369709014892578,-0.15668347477912903,-0.48421671986579895,0.7461018562316895,0.4847852289676666,-0.5157978534698486,0.6089175343513489,-0.06863506883382797,0.6623156666755676,0.3506501615047455,-0.23970338702201843,1.3757261037826538,0.24519728124141693,0.3766104578971863,0.34393593668937683,-0.4007093012332916,-0.46767935156822205,-0.9936726689338684,0.5676119923591614,0.6539759039878845,0.5281356573104858,-0.8285414576530457,0.9467968344688416,0.512235701084137,6.205699920654297,0.17964863777160645,-0.2995794117450714,-0.9064069390296936,0.8691268563270569,-0.8304623961448669,0.08734709024429321,0.01316306833177805,1.2356263399124146,0.4193289279937744,-0.41859671473503113,-0.42198771238327026,-0.5385008454322815,0.36631420254707336,0.4675295650959015,-1.159087061882019,-1.0840816497802734,0.710635781288147,-1.3635460138320923,0.2004004865884781,-0.8785640597343445,-0.5544201731681824,-0.17345623672008514,1.5385063886642456,-0.1824655830860138,0.7988689541816711,0.10359670221805573,0.30055075883865356,-0.10120060294866562,0.9264342188835144,-0.08200297504663467,-0.11403495073318481,-0.47915640473365784,-0.9860774874687195,-0.4562811851501465,-0.17222538590431213,-0.32652637362480164,1.224143147468567,0.08973173797130585,-1.2279232740402222,-1.1428816318511963,-0.3989010155200958,-0.370765745639801,-0.6666457056999207,-0.3377546966075897,0.9940035939216614,-1.153123140335083,-0.3126361668109894,-0.16023024916648865,-0.7119390368461609,-1.1199454069137573,-0.3505874276161194,0.32630810141563416,1.158528208732605,0.25375184416770935,-1.1360087394714355,-0.44708138704299927,0.019463544711470604,0.1919679343700409,-0.2495487630367279,-0.7219204306602478,0.15973177552223206,0.15817244350910187,-1.1534663438796997,-0.37709400057792664,-0.5067721605300903,-0.09889789670705795,0.9031351208686829,0.05547117069363594,0.2588669955730438,0.8931804299354553,-1.3859895467758179,-0.8250361680984497,0.7368003129959106,-0.03235969692468643,0.2744041085243225,0.18827401101589203,-1.0457125902175903,0.6596142053604126,0.6442399024963379,-0.09894532710313797,-0.9763768315315247,-0.30752190947532654,-0.6942685842514038,-0.2404772788286209,-0.1085982471704483,-0.6145074963569641,-0.7664498686790466,-0.7435756325721741,0.477243572473526,0.10569386184215546,0.12083379179239273,-1.1983355283737183,-0.4733187258243561,0.15494564175605774,-0.5411202311515808,0.2921973168849945,1.077858805656433,-0.8513641357421875,-0.1733774095773697,-0.009536011144518852,-0.12787577509880066,0.5129239559173584,-0.282955139875412,-0.13001751899719238,-0.572876513004303,0.6095629334449768,-0.43965667486190796,-1.2204612493515015,0.17870022356510162,-0.1893472820520401,0.3877820372581482,-0.8170748949050903,0.71442049741745,0.3760280907154083,-0.47244301438331604,0.06485117971897125,-0.6895230412483215,0.22009745240211487,-0.7760981321334839,-0.38591402769088745,-0.03472735360264778,-0.31244227290153503,0.2957790493965149,-0.9964895844459534,-0.5473514795303345,0.6123924851417542,-0.2511867880821228,-0.8154768347740173,-1.577586054801941,-0.2584627866744995,0.0530182346701622,-0.3631095290184021,-0.8850422501564026,0.36268725991249084,-0.39484819769859314,0.6914284825325012,0.2870795428752899,0.6362385749816895,-0.5823261141777039,-0.8835052251815796,-0.13011857867240906,-1.0067330598831177,0.6776752471923828,1.6995559930801392,-0.5604248642921448,-0.0203075148165226,-0.45848241448402405,0.5976525545120239,0.3214084208011627,-0.6156601309776306,0.010605737566947937,0.24601200222969055,-0.2573373317718506,1.0449762344360352,0.30682700872421265,1.202987790107727,0.506129801273346,0.19077672064304352,-0.4556219279766083,-1.099493145942688,0.17484526336193085,0.6521602272987366,-0.21132545173168182,0.6665617823600769,-1.178471326828003,1.012100100517273,0.01747792959213257,0.5112209320068359,0.8791051506996155,-0.442397803068161,-0.7398414015769958,0.1581886261701584,-0.24122010171413422,-0.3036032021045685,0.24077542126178741,-0.4963820278644562,-0.15936186909675598,-0.4839276671409607,0.07373825460672379,0.15987958014011383,-0.28994038701057434,0.0017468780279159546,-0.26233652234077454,0.3729775846004486,-0.3724052309989929,0.7448704838752747,-0.9129047989845276,-0.258497953414917,-0.6519159078598022,0.5356184244155884,1.193241834640503,0.12600360810756683,0.12026528269052505,-0.1445348709821701,-0.17112872004508972,-0.36479049921035767,-0.21575666964054108,-0.10598087310791016,-0.3066283166408539,-0.2787514925003052,-0.11113851517438889,-1.0467482805252075,-0.03559938818216324,-0.9802819490432739,-0.3342103064060211,-0.22652781009674072,0.24513112008571625,-0.8534258604049683,0.3332751393318176,0.3424302041530609,0.8539499044418335,-0.25175175070762634,-0.1695437729358673,0.27488574385643005,0.016132378950715065,-0.7644345164299011,0.1865558922290802,0.24402864277362823,0.4080839455127716,0.26857292652130127,0.3587067425251007,-0.3645055294036865,0.26245662569999695,0.21186961233615875,0.9407760500907898,0.28638195991516113,0.13872411847114563,-1.1295374631881714,-0.035646144300699234,0.7358245253562927,0.10563598573207855,-0.051411211490631104,-0.680521547794342,-0.7576221823692322,0.039927683770656586,-0.030639048665761948,-0.4359413683414459,-0.6076947450637817,-0.9043048620223999,0.2197735607624054,0.3163585066795349,0.44353485107421875,2.6561572551727295,-0.08384362608194351,-0.297089546918869,0.5568680763244629,-0.9405399560928345,-0.41314592957496643,-0.06623782962560654,-0.8750168681144714,0.038764722645282745,0.45225730538368225,0.2507534325122833,0.6912888884544373,-0.09464798122644424,0.5751896500587463,-0.47179555892944336,-0.37371626496315,0.08187907189130783,-1.1036769151687622,0.753860354423523,-0.16009804606437683,-0.6257186532020569,-0.23411491513252258,-0.1314982920885086,-1.0069334506988525,0.11477281153202057,0.25567033886909485,0.3561558127403259,0.6582061052322388,1.3276523351669312,1.0121744871139526,1.0183788537979126,0.48455676436424255,1.205688238143921,0.44616344571113586,0.24628695845603943,1.0418672561645508,0.9882927536964417,-0.7930888533592224,-0.29770779609680176,0.7169598340988159,0.7644096612930298,1.696358561515808,0.12976107001304626,-0.2591807544231415,-0.6207504868507385,0.14765405654907227,0.8406973481178284,-0.39812955260276794,-0.48456740379333496,-0.941591203212738,-0.2907460629940033,0.15332213044166565,0.27544090151786804,-0.32870203256607056,0.08063509315252304,-0.9007374048233032,-0.36213141679763794,0.49353712797164917,-1.0334819555282593,-0.014192082919180393,0.05352902412414551,0.06242937222123146,0.2749410569667816,-0.6889273524284363,0.7915364503860474,1.382020354270935,-0.546327531337738,-0.24568508565425873,-0.017072264105081558,0.8975743055343628,-0.04359207674860954,-0.5418493151664734,1.4584605693817139,0.29620102047920227,0.20127472281455994,-1.1004788875579834,1.1813459396362305,0.21086950600147247,0.7883961796760559,-0.05586801841855049,0.4613185226917267,0.648310661315918,-1.0370792150497437,-0.4823373854160309,0.1466970145702362,-0.9175276160240173,1.289697527885437,-0.5059159398078918,-0.02163437195122242,0.4772936999797821,0.13502134382724762,0.10893996804952621,-0.02798197790980339,-0.14120401442050934,-0.7090033888816833,-0.09093649685382843,-1.753030776977539,-0.28155237436294556,0.7232576012611389,0.2038271427154541,0.10846348851919174,0.15885566174983978,0.9904949069023132,0.5657376050949097,-0.15260565280914307,0.39597830176353455,0.09235649555921555,-0.8025262951850891,-0.6740671992301941,-0.45152631402015686,0.15151028335094452,-0.5782585144042969,0.19617962837219238,0.058297961950302124,-0.16911698877811432,0.12227809429168701,-1.1401636600494385,0.48145124316215515,0.4258110821247101,-0.8084756135940552,0.5492460131645203,-0.25864148139953613,-0.23052313923835754,-0.6592627167701721,0.32209768891334534,-1.8147965669631958,-0.474582701921463,0.43929266929626465,0.9053329825401306,0.7772269248962402,0.03410540893673897,-0.24712495505809784,0.41749143600463867,-0.26928576827049255,0.31339946389198303,0.028065038844943047,0.9890974164009094,-0.44388866424560547,0.5997099280357361,-0.4603591561317444,0.3547385334968567,1.4840341806411743,-0.6351183652877808,0.42625027894973755,-0.7275466322898865,-0.4521348178386688,-1.5241672992706299,0.058674830943346024,0.5547752976417542,-0.11832644045352936,0.8098444938659668,-0.2818610668182373,-0.20850303769111633,-1.1426960229873657,-0.05988172069191933,0.7718434929847717,1.2442594766616821,0.5289775729179382,-0.4313012659549713,-0.9896671175956726,-0.012042880058288574,-0.7077431678771973,0.12297307699918747,-0.5544127821922302,0.45235440135002136,-0.44453856348991394,-0.9708427786827087,0.7551937103271484,-0.19804270565509796,0.34608155488967896,1.6661375761032104,-0.36009666323661804,-0.4896671175956726,-0.5424214601516724,1.317615270614624,-0.08423717319965363,-0.17125900089740753,0.7611023783683777,0.8835111856460571,-0.48037147521972656,-0.27077314257621765,-0.43617358803749084,1.2427173852920532,-0.3531494736671448,-1.0773961544036865,-0.6107911467552185,0.8619478940963745,-0.6072909235954285,-0.20692183077335358,-0.17758230865001678,-0.36724090576171875,0.5605141520500183,-0.4557349979877472,-0.5249633193016052,0.31979915499687195,-0.47805055975914,-1.8788217306137085,-0.4154564440250397,1.5306020975112915,0.015479586087167263,-0.13008944690227509,-0.6883378624916077,-0.24753247201442719,0.4665422737598419,-1.8523625135421753,0.9479907155036926,0.691737174987793,-0.5840218663215637,0.28516629338264465,0.12467215210199356,-0.03483372554183006,-1.0444610118865967,0.005288754124194384,0.3996627926826477,-0.03001350536942482,1.215628743171692,0.18670007586479187,-0.3712849020957947,-0.2946054935455322,0.5540713667869568,-0.15919364988803864,0.49659204483032227,-0.5765485167503357,-0.12424834072589874,-0.1256193071603775,0.34270936250686646,0.4926934242248535,-0.15547628700733185,0.5898733139038086,0.8413854241371155,-0.5769823789596558,0.45015498995780945,0.21575824916362762,-0.1949256807565689,0.4553467333316803,1.4813207387924194,-1.1403392553329468,-0.20150132477283478,0.4779489040374756,-0.1530890017747879,-0.3760153353214264,0.8181034326553345,-1.0308951139450073,0.37751010060310364,-1.2903969287872314,-1.3498378992080688,-0.4537864625453949,-0.5630099177360535,1.2241085767745972,-0.8967321515083313,0.22193752229213715,-1.135604977607727,-0.23144139349460602,-0.31987497210502625,1.2027686834335327,-0.11440461128950119,-1.6468746662139893,0.8795199990272522,-0.41378507018089294,0.42868638038635254,-0.23489096760749817,0.324205219745636,-1.4596720933914185,-0.3669913709163666,0.5943787693977356,-0.3477131724357605,0.6264639496803284,-0.21347875893115997,0.11572062969207764,-1.097939372062683,-0.5588022470474243,0.6662203073501587,-0.4518408179283142,-0.49844518303871155,0.43972593545913696,0.007790873292833567,-1.3984696865081787,-0.11220186948776245,0.70350182056427,0.314160019159317,-0.30072906613349915,-0.706821620464325,-0.4230189323425293,-0.6225741505622864,0.26218950748443604,0.5146000981330872,0.18876351416110992,0.8386269211769104,0.23924438655376434,0.24905240535736084,0.08779782801866531,-0.518436074256897,0.44892778992652893,-0.9769865870475769,0.9801388382911682,0.19578328728675842,-0.34570857882499695,-0.0693383663892746,-0.19045278429985046,1.2618348598480225,0.48324480652809143,0.3812315762042999,0.324099600315094,-0.6927453279495239,-0.15553610026836395,0.835305392742157,0.642917811870575,0.45273199677467346,0.22228379547595978,1.1627976894378662,0.12146474421024323,-0.7109533548355103,1.7619649171829224,-0.07762619107961655,0.4769858717918396,0.14360035955905914,-1.4717309474945068,0.7066775560379028,1.5033801794052124,-0.6289511919021606,0.2806159555912018,-0.2051655799150467,-0.9667419791221619,0.23085899651050568,0.5434554219245911,0.7447602152824402,0.5415276885032654,-0.48468881845474243,-0.8282933235168457,0.3875649869441986,-0.40981951355934143,-0.9249709248542786,1.169667363166809,1.1850099563598633,0.9564076066017151,-0.4689408242702484,0.37945690751075745,0.8240247964859009,0.28802505135536194,0.2318928986787796,-0.04754745960235596,0.708986759185791,0.0305455531924963,-0.022934820502996445,-0.5268754959106445,0.8526889085769653,1.173460841178894,0.4552220404148102,-0.8132721185684204,0.6007777452468872,0.10756505280733109,0.039531875401735306,0.9437265992164612,0.25729891657829285,0.2206597626209259,-0.2778460681438446,0.564242422580719,-0.5584037899971008,0.38256731629371643,-1.3771647214889526,-0.23349972069263458,0.41487187147140503,0.9794707894325256,-0.25642988085746765,-1.318642497062683,-0.8618022799491882,0.14537706971168518,-0.5366196632385254,1.4602652788162231,-1.2555429935455322,-0.593861997127533,-0.6432135701179504,0.568175196647644,-0.8961955308914185,0.08029492199420929,1.5162639617919922,1.3542072772979736,-0.25798019766807556,0.6849989295005798,-1.0795567035675049,0.3870997428894043,-0.9572779536247253,0.720069944858551,0.32925906777381897,0.06001025438308716,-0.03610371798276901,-1.6400489807128906,-0.2505847215652466,1.2747706174850464,-0.9557420015335083,-0.6234267950057983,-0.3097229599952698,-1.1966803073883057,0.6190375685691833,-0.1470983922481537,0.5514560341835022,-0.2866491973400116,0.597439169883728,-0.19421400129795074,0.6676111221313477,-1.0090677738189697,2.6685352325439453,-0.0776020959019661,0.7851082682609558,-0.09770768880844116,-1.0210450887680054,0.8387726545333862,0.9654936194419861,-0.08691970258951187,-0.595326840877533,-0.3550753891468048,-0.11177264899015427,-0.1471196711063385,-0.02220485545694828,-0.3550775647163391,-0.6632953882217407,-0.7508983612060547,-0.7135671973228455,0.31867584586143494,-1.2461978197097778,0.13422127068042755,0.1906871497631073,-0.48376554250717163,-0.43893975019454956,0.5823253989219666,0.3170661926269531,0.6251915097236633,-1.061818242073059,0.7928206324577332,-0.17399512231349945,-0.02413882687687874,1.0688568353652954,0.3442021310329437,1.796317219734192,0.39448049664497375,0.10370159894227982,0.3273486793041229,-0.9359150528907776,-0.8385778665542603,-1.261798620223999,0.3951174318790436,1.218258023262024,-0.17817051708698273,0.399299681186676,1.1199451684951782,0.46816855669021606,0.027166489511728287,0.6064011454582214,0.32896023988723755,-0.23404541611671448,-1.37547767162323,-0.9163504242897034,-0.3632964491844177,0.2134859263896942,0.5273730754852295,0.9582116007804871,-0.0008642202010378242,0.5967943072319031,-0.1686803251504898,0.053981006145477295,1.5568616390228271,-0.535121500492096,0.569788932800293,-1.17081618309021,0.5799431204795837,-0.07698620110750198,-0.5558425784111023,-0.6515501141548157,0.5787015557289124,-1.0200389623641968,-0.7202025651931763,-1.4898821115493774,-0.10708187520503998,-0.059374745935201645],k: 10,num_candidates: 100},_source: [title]
}向量搜索的召回 [{_index: my_index,_id: 0694ae411b304dd0020569fc11c6d255,_score: 0.931366,_source: {title: 除了北京。哪里还合适作为新中国首都。}},{_index: my_index,_id: 1a13cd4a80dceb474ffa249bcd858b0d,_score: 0.931366,_source: {title: 除了北京。哪里还合适作为新中国首都。}},{_index: my_index,_id: 1260eb87c371618f23e855b5bd21bc59,_score: 0.931366,_source: {title: 除了北京。哪里还合适作为新中国首都。}},{_index: my_index,_id: 672061a2a6e1ba2f4588f64b27e7a51f,_score: 0.9063271,_source: {title: 北京的初回忆}},{_index: my_index,_id: 82de4fdefa7b0ae5737a7f2745691441,_score: 0.8953391,_source: {title: 一千中国人离开中国迁徙海外租借土地999年建起一座中国城}},{_index: my_index,_id: 5be4873b15d2faa36c98b237fd20dcd3,_score: 0.8938775,_source: {title: 第一次来首都北京但不是为了玩乐而且千里求医愿一切顺利}},{_index: my_index,_id: 6cdb573966fad2703fd31546a1c2bd6b,_score: 0.89284897,_source: {title: 平安到家乡的城市}},{_index: my_index,_id: 577415939eb39c9b4829fe023ff9e620fw,_score: 0.89217013,_source: {title: 北京首都开发股份有限公司}},{_index: my_index,_id: f2d5f9ab7d92bbdb142f5ccb03feaaef,_score: 0.8895248,_source: {title: 北京城儿那些事儿}},{_index: my_index,_id: b9de2d92d19f8ad875c2cafde08b360fbjsj,_score: 0.8891097,_source: {title: 好像只有我们中国人最爱一股脑的往城里挤。住着高楼大厦像蚂蚁}}]
