How to scrape dynamic website? - screen-scraping

I am a novice website scraper, trying to get a usable list of all past events on Hackathon.io. Any advice would be greatly appreciated. I have been using the chrome scraper extension but it comes up empty/blank for this website. Thanks
http://www.hackathon.io/events

The more button can be enumerated like so: http://www.hackathon.io/events?location=&page=61&term=&time=past&utf8=%E2%9C%93 ; you would just iterate through those and grab the links using the appropriate xpaths.

Related

Create a thumbnail of document after uploading?

I am making a tiny website with reactJS where users can upload documents and share them,since multiple extensions are being uploaded,I managed to make the viewing work by just using google docs viewer.
but now i want to have thumbnails of those files,maybe of the first page so that it shows up un the search feature.
seeing that theres many extensions flying around i cant figure out how to approach this,what can i use to solve this problem ?
thank you.

What is the best way to scrape content rendered by a Chrome Extension?

I want to scrape the content rendered/showed by a Chrome Extension - not a website. The webpage URL triggers a request to this specific Chrome Extension, and they render content that comes from their database.
I'm trying to automate the current workflow we have.
What are the best tools and ways for this task?
Thanks & Love.
I guess it depends on what you are trying to accomplish. Do you need to interact with the content after it is rendered by the extension? You may be able to use something like Cypress: https://github.com/cypress-io/cypress

JSON performance tips. Drupal backend and IONIC Frontend

I am creating a hybrid app (IONIC + AngularJS). Using Drupal as my Backend. I am using Drupal services and getting the JSON data.
I would like to improve the speed/performance of getting the data. Most of the data will be images.
On the first page, i just want a list of the user's firstnames. When you click on a name you get the details page for that user. This page contains a lot of images.
What is the best way of doing this?
If my list contains a 100 people with profile pictures. I don't want to download all this information. Just the information of the person that was selected from the list. Is this possible? Is this a good idea? I am sorry if this is dumb questions. I am a beginner.
Do you have a better suggestion on how to do this?
Thanks in advance
JSON
[{"firstName":"John", "Image1":"John.jpg", "Image2":"John2.jpg", "Image3":"John3.jpg"},
{"firstName":"Jane", "Image1":"Jane.jpg", "Image2":"Jane2.jpg", "Image3":"Jane3.jpg"},
{"firstName":"Sam", "Image1":"Sam.jpg", "Image2":"Sam2.jpg", "Image3":"Sam3.jpg"}]
Here are few links you might want to go through for performance
https://www.sitepoint.com/5-ionic-app-development-tips-tricks/
http://julienrenaux.fr/2015/08/24/ultimate-angularjs-and-ionic-performance-cheat-sheet/
http://www.gajotres.net/speed-up-your-ionic-application-using-these-techniques/
Regards.

PDF generation from AngularJS

I made an AngularJS page with multiple tables, and I need a pdf generation solution. I'm working in a Phonegap context.
The tools I found until now were not able to handle the CSS in my page (like JSpdf).
Do you know some great tools to solve my problem ?
Thanks in advance.
You can try phantomJS. http://phantomjs.org
It provides print to pdf function too.

How do I convert my HTML/CSS site to be used on Mobile/Tablet

I made a website in complete HTML/CSS and a little bit of Javascript and Flash. I would like to know if it is possible to have it be good for Mobile/Tablet.
I am working in Dreamweaver and i clicked FILE > CONVERT > XHTML MOBILE 1.0 but got this error message "couldt fix 4 elements without alt attribute.
If anyone could help I would greatly appreciate it, thank you!
Use response CSS.
Twitter Bootstrap is great for this sort of thing. Divshot is a good tool for constructing user interfaces with Twitter Bootstrap.

Resources