xml parsing - Scraping HTML-page using readHTMLTables in R -

-

i'd scrape webpage: http://unbisnet.un.org:8080/ipac20/ipac.jsp?&menu=search&aspect=power&npp=50&ipp=20&spp=20&profile=bib&index=.tw&term=%22draft+resolution%22&index=.aw&term=netherlands

i need create dataframe containing 5 columns (title, imprint, enhanced title, un document symbol, publication date) each resolution. tried using readhtmltables, can't seem figure out. website contains many tables. when run code list 394 objects, me seems empty.

u <- "http://unbisnet.un.org:8080/ipac20/ipac.jsp?&menu=search&aspect=power&npp=50&ipp=20&spp=20&profile=bib&index=.tw&term=%22draft+resolution%22&index=.aw&term=netherlands" tables = readhtmltable(u) names(tables) length(tables) tables[[1]]

you have change approach. that's horrible site many, many nested <table>s. here's 1 way deal it, you'll have roll sleeves fields may want. please research xpath queries or post specific follow-up "how to"'s separate questions (i.e. how target specific xpath element) vs tack on post.

we'll use hadleyverse:

library(rvest) library(dplyr)

this reads in html page:

doc <- html("http://unbisnet.un.org:8080/ipac20/ipac.jsp?&menu=search&aspect=power&npp=50&ipp=20&spp=20&profile=bib&index=.tw&term=%22draft+resolution%22&index=.aw&term=netherlands")

at point in nested structure there's <table> 1 <tr> , 2 <td> elements have field , data field. 1 way data value field target label field (which in <a> tag), go 1 level (to <td>) take it's next sibling (the value <td>). example, title: values:

titles <- xml_text(             html_nodes(doc,                xpath="//td/a[starts-with(., 'title:')]/../following-sibling::td[1]"))

you can same these 3 fields:

imprints <- xml_text(html_nodes(doc, xpath="//td/a[starts-with(., 'imprint:')]/../following-sibling::td[1]")) un_doc_sym <- xml_text(html_nodes(doc, xpath="//td/a[starts-with(., 'un doc')]/../following-sibling::td[1]")) pub_date <- xml_text(html_nodes(doc, xpath="//td/a[starts-with(., 'publication date')]/../following-sibling::td[1]"))

and wrap them in data_frame / data.frame:

unbisnet <- data_frame(titles, imprints, un_doc_sym, pub_date)  glimpse(unbisnet)  ## observations: 50 ## variables: ## $ titles     (chr) "draft resolution [on establishment of internati... ## $ imprints   (chr) "[new york] : un, 29 july 2015", "[new york] : un, ... ## $ un_doc_sym (chr) "s/2015/562", "a/69/l.80", "a/69/l.78", "a/69/l.76"... ## $ pub_date   (chr) "20150729", "20150715", "20150630", "20150625", "20...

targeting of others won't easy it's doable.


Comments

Post a Comment

Popular posts from this blog

html - Firefox flex bug applied to buttons? -

-
i'm trying apply flex button on firefox i'm having problem on chrome doesn't happens. this css: .button { display: inline-flex; align-items: center; } as can see in fiddle, "caret" word moved new line, happens if element <button> tag, <div> works expected. http://codepen.io/anon/pen/xblmgx is firefox bug or there css i'm missing? nevermind, looks "mr firefox" doesn't idea... <button> not implementable (by browsers) in pure css, bit of black box, perspective of css. means don't react in same way e.g. <div> would. this isn't specific flexbox -- e.g. don't render scrollbars if put "overflow:scroll" on button, , don't render table if put "display:table" on it. stepping further, isn't specific <button> . consider <fieldset> , <table> have special rendering behavior: data:text/html,<fieldset style="d
Read more

html - Missing border-right in select on Firefox -

-
Image
when put select div width , padding-left, options have no border on right. more, happend if select have width 118px , parent have 15px of padding left! have idea, going on? my code: <!doctype html> <html> <head lang="en"> </head> <body> <div style="width: 1000px; padding-left: 15px;"> <select style="width: 118px;"> <option>a</option> <option>b</option> <option>c</option> </select> </div> </body> </html> and result: here plunker: click it happend on firefox on windows. tested on 40.0.2 version -webkit-box-sizing: border-box; -moz-box-sizing: border-box; -ms-box-sizing: border-box; box-sizing: border-box; source: https://stackoverflow.com/a/11390432/4431269 source of problem: https://bugzilla.mozilla.org/show_bug.cgi?id=924068 and use of google ! edit: (source of problem, explained) after rea
Read more

c# - two queries in same method -

-
i trying execute 2 queries in same method gives me exception. can red of exception declaring new command there way use same command? string id="hi"; connection.open(); oledbcommand command1 = new oledbcommand(); command1.connection = connection; string query1 = "select * products category='" + combobox1.text + "' , subcategory = '" + combobox2.text + "' , sizes='" + combobox3.text + "'"; command1.commandtext = query1; oledbdatareader reader = command1.executereader(); while (reader.read()) { id = reader[0].tostring(); } textbox1.text = id; string query = "insert category_in (category_id,amount,amount_in) values ('"+ id+"' ,500,300)"; command1.commandtext = query; command1.executenonquery(); messagebox.show("saved"); connection.close();
Read more