做网站的自我介绍,网页版有意思的游戏排行榜,荆州 网站建设,网站建设好后为什么要维护我正在做一个scrapy spider#xff0c;我必须发送一个post请求循环才能转到下一个页面#xff0c;问题是它只发送一个post请求。querystring更改每个页面的元素“currentPage”#xff0c;因此我必须为每个页面更改此键的值并发送post。但是#xff0c;正如我之前所说…我正在做一个scrapy spider我必须发送一个post请求循环才能转到下一个页面问题是它只发送一个post请求。querystring更改每个页面的元素“currentPage”因此我必须为每个页面更改此键的值并发送post。但是正如我之前所说它在第一个Post请求之后停止。在import scrapyheaders {accept: */*,origin: http://www.**********.com,x-requested-with: XMLHttpRequest,user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36,referer: http://www.**********.com/venta/,accept-encoding: gzip, deflate,accept-language: en-US,en;q0.8,es;q0.6,cookie: G_ENABLED_IDPSgoogle; cookieInterestedProject416; visid_incap_434661wjkf7tUQPKDjpmWXz/BKSBz1kAAAAAQUIPAAAAAAA7bs2fXOSL0JmeVSXo337M; incap_ses_223_4346619zyRHwEdwGxtE8Ly00EYAxQw/VkAAAAAq7gkFJrJjsGdCgrRTwOfvg; s_vnum1512243236606%26vn%3D3; __utmzother; s_cmNatural%20Searchwww.google.com.co; s_v10%5B%5B%27Natural%2520Search%27%2C%271509651236616%27%5D%2C%5B%27Natural%2520Search%27%2C%271509651249121%27%5D%2C%5B%27Natural%2520Search%27%2C%271509765142570%27%5D%2C%5B%27Natural%2520Search%27%2C%271509765184463%27%5D%5D; s_v8%5B%5B%27natural%2520search%253A%2520google%253A%2520keyword%2520unavailable%27%2C%271509651236618%27%5D%2C%5B%27natural%2520search%253A%2520google%253A%2520keyword%2520unavailable%27%2C%271509651249123%27%5D%2C%5B%27natural%2520search%253A%2520google%253A%2520keyword%2520unavailable%27%2C%271509765142572%27%5D%2C%5B%27natural%2520search%253A%2520google%253A%2520keyword%2520unavailable%27%2C%271509765184465%27%5D%5D; ; s_cctrue; _gaGA1.2.701497075.1509651237; _gidGA1.2.1068485902.1509765143; NSC_nfuspdvbesbep-wtffffffff0975c87745525d5f4f58455e445a4a4229a2; OX_sd1; OX_plgpm; gpv_pnmetrocuadrado%3A%20buscar%3A%20resultados%20inmuebles%3A%20nuevo%20y%20usado; s_invisittrue; s_nr1509765213941-Repeat; s_lv1509765213944; s_lv_sLess%20than%207%20days; s_sqeltiempometrocuadradoprod%2Celtiempoglobal%3D%2526pid%253Dmetrocuadrado%25253A%252520buscar%25253A%252520resultados%252520inmuebles%25253A%252520nuevo%252520y%252520usado%2526pidt%253D1%2526oid%253Dhttp%25253A%25252F%25252Fwww.metrocuadrado.com%25252Fventa%25252F%252523%2526ot%253DA; madicionales; mbarrio; mciudad; mgrupo; mgrupoid; mnrobanos; mnrocuartos; mnrogarajes; msector; mubicacion; mvalorarriendo; mzona; orderBy; selectedLocationCategory; selectedLocationFilter; sortType; writtenFiltersmnrogarajes%3Bmnrobanos%3Bmnrocuartos%3Bmtiempoconstruido%3Bmarea%3Bmvalorarriendo%3Bmvalorventa%3Bmciudad%3Bmubicacion%3Bmtiponegocio%3Bmtipoinmueble%3Bmzona%3Bmsector%3Bmbarrio%3BselectedLocationCategory%3BselectedLocationFilter%3Bmestadoinmueble%3Bmadicionales%3BorderBy%3BsortType%3Bmestadoinmueble%3BcompanyType%3BcompanyName%3Bmidempresa%3Bmgrupo%3Bmgrupoid%3B; m2-srvffffffff0975c82e45525d5f4f58455e445a4a4229a2; mtiponegocioventa; mtipoinmueble; mvalorventa; marea; mtiempoconstruido; companyType; companyName; midempresa; mestadoinmueble,cache-control: no-cache,postman-token: 2e5f00b9-7c7c-32ed-1bdd-63cf2fed3cd8}querystring {:,mnrogarajes:,mnrobanos:,mnrocuartos:,mtiempoconstruido:,marea:,mvalorarriendo:,mvalorventa:,mciudad:,mubicacion:,mtiponegocio:venta,mtipoinmueble:,mzona:,msector:,mbarrio:,selectedLocationCategory:,selectedLocationFilter:,mestadoinmueble:,madicionales:,orderBy:,sortType:,companyType:,companyName:,midempresa:,mgrupo:,mgrupoid:,currentPage:2,totalPropertiesCount:115747,totalUsedPropertiesCount:113926,totalNewPropertiesCount:1821,sfh:1}url http://www.*******.com/search/list/ajaxnum 0class HouseseSpider(scrapy.Spider):name hoimomstart_urls [http://www.********.com/venta/]def parse(self,response):for num in range(2,100):for href in response.xpath(.//a[classdata-details-id and itempropurl]/href).extract():yield scrapy.Request(url href ,callback self.parsei)querystring[currentPage] str(num)yield scrapy.Request(url http://www.*********.com/search/list/ajax,methodPOST,headersheaders,metaquerystring)def parsei(self, response):yield {latitude: response.xpath(//input[idlatitude]/value).extract(),longitud: response.xpath(//input[idlongitude]/value).extract(),precio de arriendo: response.xpath(.//dl/dt[h3/text()Valor de arriendo]/following-sibling::dd[1]/h4/text()).extract_first(),precio de venta: response.xpath(.//dl/dt[h3/text()Valor de venta]/following-sibling::dd[1]/h4/text()).extract_first(),Barrio_com: response.xpath(.//dl/dt[h3/text()Nombre común del barrio ]/following-sibling::dd[1]/h4/text()).extract_first(),Barrio_cat: response.xpath(.//dl/dt[h3/text()Nombre del barrio catastral]/following-sibling::dd[1]/h4/text()).extract_first(),Estrato: response.xpath(.//dl/dt[h3/text()Estrato]/following-sibling::dd[1]/h4/text()).extract_first(),id: response.xpath(//input[idpropertyId]/value).extract_first(),Habitaciones: response.xpath(.//dl/dt[h3/text()Habitaciones]/following-sibling::dd[1]/h4/text()).extract_first(),Parqueadero: response.xpath(.//dl/dt[h3/text()Parqueadero]/following-sibling::dd[1]/h4/text()).extract_first(),Tipo de calentador: response.xpath(.//dl/dt[h3/text()Tipo de calentador]/following-sibling::dd[1]/h4/text()).extract_first(),Cuarto de servicio: response.xpath(.//dl/dt[h3/text()Cuarto de servicio]/following-sibling::dd[1]/h4/text()).extract_first(),Tipo de acabado piso: response.xpath(.//dl/dt[h3/text()Tipo de acabado piso]/following-sibling::dd[1]/h4/text()).extract_first(),Area_Cons: response.xpath(.//dl/dt[h3/text()Área construida]/following-sibling::dd[1]/h4/text()).extract_first()}
