What is the techniques to implement Visual Web Scraper? - screen-scraping

I'm going to build a visual web scraper. The most important feature the software required is "visual" like http://mozenda.com/.
The software create a tool like web-browser not only allow user to browse a webpage, perform some tasks as authenticate, click links, make searching, ... but also can track all these tasks.
Does anyone know the techniques to implement this?
Thanks.

import.io is way to go! It's free ATM

Did you checked Selenium?

With IRobotSoft Visual Web Scraper, you are able to visually record web scraping actions.

I don't know what kinds of tasks you are looking for... but this sounds like a Firefox extension you are trying to build and automate.
I would look at those docs to see how you can attach to the main browser window and track those click and keyboard actions.

Related

Alternative for Workbench

Hope everyone will know, In recent days Workbench website is not functioning well and it's provide Application Error during SOQL Execution or any other operation.
So May I know, Is there any other alternative website is available for Workbench?
I know, we can use Developer Console OR Report inside the Salesforce OR SFDX Query Editor are able to use for alternative, But I like to know is there any website available to do that?
Thanks.
Use chrome extension Salesforce Inspector.

Create Better Looking Buttons in VBA without using Image

Is there anyway to create more rounded/professional looking buttons using vba. I found a couple places that suggested using images, which is not a terrible Idea, but I was wondering if anyone had a clue if there is a downloadable software, add-in, or anything. The ones currently available make my program look like an Mid 90's application. I really appreciate the help. I mean look at it!!
Office VBA uses the Microsoft Forms Engine which is a lightweight (drawn) implementation of a GUI toolkit. Because of this the usual techniques used to customise button appearance on windows (owner-drawing/theming) don't apply and your stuck with what you've got.
If you don't want to use images you need to find an ActiveX control that does the job, something like http://www.arcadiahome.com/products/components/activex/powerbutton.htm .

Creating an online Asset Library/Catalogue? Help

I've been asked to look in to creating and online database for sorting flash banners. So its kind of like a big resource library where our client can log on search and browser for old/existing banner creatives.
Does anyone have any recommendations on what I should do/look in to. CMS Framesworks etc.
I'm pretty sure I could use Wordpress for this job via custom post types etc. But I think there's probably a better solution out there. Drupal? Joomla? Expression Engine? Or would it be better to just create a basic cms from scratch.
Features needed:
Kick arse search functionality (am guessing the client will likely try to search for creative by year, month, campaign, banner type.
Smart navigation
Sharing is convenient
Must be able to demo working demos of expanding banners as well as non-expanding
CMS so new ads can be easily added to the library.
Thanks in advance for you knowledgeable insights :P
cheers
Although basic Joomla has own extension for this purpose, here:
http://extensions.joomla.org/extensions/ads-a-affiliates/banner-management , you have got a whole set of advanced extensions which do the job for you in Joomla. Read opinions and choose your favourite

Hows Mozenda Screen Scraper coded?

I want to know how is the Mozenda Screen Scraper coded?
http://www.mozenda.com/screen-scraper
I shows a browser where user can select the fields he wants to scrap and it creates a crawl script out of it.
The crawl script generation part is clear to me, I want to know does it record the user actions which the user is doing in the browser.
I want to do something similar using C#.
I guess they run something like iMacros on the backend. I even noticed the iMacros browser icon in some of their screenshots ;-)
If you want to code something like this yourself in C#, the IE webbrowser control is the best starting point!
#yc08m, I can tell you for sure that they are not using iMacros. It is all coded with their own proprietary code.
I realize this is an extremely outdated post, but they seem to rely heavily on XPath, so first things first you would need to load the page into an object that you can query. I'd recommend using something like Html Agility pack to accomplish this.

Tools for automating SilverLight application

I want to automate the application develoepd in silverlight. Can you let me know whether there are any open source tools are available which can be used for automating SilverLight app. Thanks.
We are doing UI testing on a Silverlight application using the UIAutomation API, and it works very nicely.
Project White is a library that sits on top of UIAutomation, and many people find it helpful. I prefer to use the UIAutomation APIs directly - it's really not as hard as you might think, as my tutorial shows.
If you're looking to perform User Interface Automation (UIA) to simulate mouse clicks, keyboard presses, etc. there is no "easy" way to get this today.
I've seen some third party tools that may work, such as Telerik's. Note that I have not used and do not endorse this, just know it is out there!
I am currently trying out Project White and finding it quite easy to use.
Before using Project White I tried out WiPFlash which was also rather easy to use. It just lacked a few features that I was after (Right Click and Drag options).

Resources