What is CasperJS?
“CasperJS is an open source navigation scripting & testing utility written in Javascript for the PhantomJS WebKit headless browser and SlimerJS (Gecko). It eases the process of defining a full navigation scenario and provides useful high-level functions, methods & syntactic sugar for doing common tasks”
What is SlimerJS?
SlimerJS is a scriptable browser. It allows you to manipulate a web page with an external Javascript script: opening a webpage, clicking on links, modifying the content… It is useful to do functional tests, page automation, network monitoring, screen capture etc.
What is the use of CasperJS?
CasperJS – Automated Testing of Web Applications with JavaScript. CasperJS is an open source browser navigation scripting and testing tool written in JavaScript for PhantomJS and SlimerJS.
Which is the best use case for CasperJS?
I find the most compelling use case for CasperJS scraping is when a site relies on a lot of JavaScript to navigate through the content; a recent project was a perfect example as it uses AngularJS, loads all the content asynchronously and uses infinite scrolling instead of pagination.
What do you use CasperJS event handler for?
So what we’re doing here is using the CasperJS event handler to detect when a new batch of content has been loaded and to keep on scrolling until the spinner is no longer shown. If the spinner is shown, we wait until it disappears again using the waitWhileVisible method.
Can a web scraper be used on Facebook?
If you use Twitter or Facebook you would have seen it – scroll to the bottom of the page quickly enough and you’ll catch the little spinner pixies doing their dance while the next batch of content is loaded. While this can make for a fluid user experience, it’s another potential pitfall for the web scraper.
What does infinite scrolling mean in web scraping?
Infinite scrolling is a navigation mechanism that is being adopted by numerous sites in place of pagination. If you use Twitter or Facebook you would have seen it – scroll to the bottom of the page quickly enough and you’ll catch the little spinner pixies doing their dance while the next batch of content is loaded.