python - I want Scrapy to run through each item once -

-

i scrapy run through each item once relevant data grouped together. puts links, headers, dates etc together. posting file more once. pretty new both scrapy , python advice grateful for.

here spider code:

from scrapy.spiders import spider  scrapy.selector import selector fashioblog.functions import extract_data  fashioblog.items import fashioblog  class firstspider(spider):    name = "first"    allowed_domains = [       "stopitrightnow.com"     ]    start_urls = [       "http://www.stopitrightnow.com"     ]     def parse(self, response):     sel = selector(response)     sites = sel.xpath('//div[@class="post-outer"]')     items= []      site in sites:         item = fashioblog()          item['title'] = extract_data(site.xpath('//h3[normalize-space(@class)="post-title entry-title"]//text()').extract())         item['url'] = extract_data(site.xpath('//div[normalize-space(@class)="post-body entry-content"]//@href').extract())         item['date'] = extract_data(site.xpath('//h2[normalize-space(@class)="date-header"]/span/text()').extract())         #item['body'] = site.xpath('//div[@class="post-body entry-content"]/i/text()').extract()         item['labellink'] = extract_data(site.xpath('//span[normalize-space(@class)="post-labels"]//@href').extract())         item['comment'] = extract_data(site.xpath('//span[normalize-space(@class)="post-comment-link"]//text()').extract())         item['picurl'] = extract_data(site.xpath('//div[normalize-space(@class)="separator"]//@href').extract())         #item['labeltext'] = extract_data(site.xpath('(//i//text()').extract())         #item['labellink2'] = extract_data(site.xpath('(//i//@href').extract())         yield item

make expressions context-specific prepending dot:

item['title'] = extract_data(site.xpath('.//h3[normalize-space(@class)="post-title entry-title"]//text()').extract())                                          ^ here

Comments

Post a Comment

Popular posts from this blog

html - Firefox flex bug applied to buttons? -

-
i'm trying apply flex button on firefox i'm having problem on chrome doesn't happens. this css: .button { display: inline-flex; align-items: center; } as can see in fiddle, "caret" word moved new line, happens if element <button> tag, <div> works expected. http://codepen.io/anon/pen/xblmgx is firefox bug or there css i'm missing? nevermind, looks "mr firefox" doesn't idea... <button> not implementable (by browsers) in pure css, bit of black box, perspective of css. means don't react in same way e.g. <div> would. this isn't specific flexbox -- e.g. don't render scrollbars if put "overflow:scroll" on button, , don't render table if put "display:table" on it. stepping further, isn't specific <button> . consider <fieldset> , <table> have special rendering behavior: data:text/html,<fieldset style="d
Read more

html - Missing border-right in select on Firefox -

-
Image
when put select div width , padding-left, options have no border on right. more, happend if select have width 118px , parent have 15px of padding left! have idea, going on? my code: <!doctype html> <html> <head lang="en"> </head> <body> <div style="width: 1000px; padding-left: 15px;"> <select style="width: 118px;"> <option>a</option> <option>b</option> <option>c</option> </select> </div> </body> </html> and result: here plunker: click it happend on firefox on windows. tested on 40.0.2 version -webkit-box-sizing: border-box; -moz-box-sizing: border-box; -ms-box-sizing: border-box; box-sizing: border-box; source: https://stackoverflow.com/a/11390432/4431269 source of problem: https://bugzilla.mozilla.org/show_bug.cgi?id=924068 and use of google ! edit: (source of problem, explained) after rea
Read more

c# - two queries in same method -

-
i trying execute 2 queries in same method gives me exception. can red of exception declaring new command there way use same command? string id="hi"; connection.open(); oledbcommand command1 = new oledbcommand(); command1.connection = connection; string query1 = "select * products category='" + combobox1.text + "' , subcategory = '" + combobox2.text + "' , sizes='" + combobox3.text + "'"; command1.commandtext = query1; oledbdatareader reader = command1.executereader(); while (reader.read()) { id = reader[0].tostring(); } textbox1.text = id; string query = "insert category_in (category_id,amount,amount_in) values ('"+ id+"' ,500,300)"; command1.commandtext = query; command1.executenonquery(); messagebox.show("saved"); connection.close();
Read more