Java HTML Parsing a Page with Infinite Scroll -

-

how can grab page's html in java if page has infinite scroll? i'm grabbing page way:

url url = new url(stringurl); urlconnection con = url.openconnection(); inputstream in = con.getinputstream(); string encoding = con.getcontentencoding(); encoding = encoding == null ? "utf-8" : encoding; string html = ioutils.tostring(in, encoding); document document = jsoup.parse(html);

but doesn't return of content associated infinite scroll section of page. how can trigger scrolling on html page jsoup document contains section?

infinite scroll describes technique page not contain content. javascript code runs in browser, sends request server addiional content , adds page. when scroll towards end of available content, javascript code repeats process: sends request , adds additional content.

therefore, need web browser javascript engine can run javascript code , produce events cause code load content.


Comments

Post a Comment

Popular posts from this blog

html - Firefox flex bug applied to buttons? -

-
i'm trying apply flex button on firefox i'm having problem on chrome doesn't happens. this css: .button { display: inline-flex; align-items: center; } as can see in fiddle, "caret" word moved new line, happens if element <button> tag, <div> works expected. http://codepen.io/anon/pen/xblmgx is firefox bug or there css i'm missing? nevermind, looks "mr firefox" doesn't idea... <button> not implementable (by browsers) in pure css, bit of black box, perspective of css. means don't react in same way e.g. <div> would. this isn't specific flexbox -- e.g. don't render scrollbars if put "overflow:scroll" on button, , don't render table if put "display:table" on it. stepping further, isn't specific <button> . consider <fieldset> , <table> have special rendering behavior: data:text/html,<fieldset style="d
Read more

html - Missing border-right in select on Firefox -

-
Image
when put select div width , padding-left, options have no border on right. more, happend if select have width 118px , parent have 15px of padding left! have idea, going on? my code: <!doctype html> <html> <head lang="en"> </head> <body> <div style="width: 1000px; padding-left: 15px;"> <select style="width: 118px;"> <option>a</option> <option>b</option> <option>c</option> </select> </div> </body> </html> and result: here plunker: click it happend on firefox on windows. tested on 40.0.2 version -webkit-box-sizing: border-box; -moz-box-sizing: border-box; -ms-box-sizing: border-box; box-sizing: border-box; source: https://stackoverflow.com/a/11390432/4431269 source of problem: https://bugzilla.mozilla.org/show_bug.cgi?id=924068 and use of google ! edit: (source of problem, explained) after rea
Read more

c# - two queries in same method -

-
i trying execute 2 queries in same method gives me exception. can red of exception declaring new command there way use same command? string id="hi"; connection.open(); oledbcommand command1 = new oledbcommand(); command1.connection = connection; string query1 = "select * products category='" + combobox1.text + "' , subcategory = '" + combobox2.text + "' , sizes='" + combobox3.text + "'"; command1.commandtext = query1; oledbdatareader reader = command1.executereader(); while (reader.read()) { id = reader[0].tostring(); } textbox1.text = id; string query = "insert category_in (category_id,amount,amount_in) values ('"+ id+"' ,500,300)"; command1.commandtext = query; command1.executenonquery(); messagebox.show("saved"); connection.close();
Read more