Is it possible to port a web scraper/web crawler over to a mobile device such as a smart phone? If so, how would I do it?
I feel like this is a loaded question... of course it is possible, but why would you want to?
If so, how would I do it?
Well, you would take an existing web crawler and port it. As a matter of fact, you might not have to do much porting: if you take a crawler like Nutch, it's already written in Java, it's available to download as a package so you can just drop it in your android phone, get root access and run the crawler. But again: why would you want to do it?
Whatever your goal is, I would think that putting a web crawler on a smart phone would be an overkill. So what's your goal?
Related
I am sort of new to app development, so this may sound like a stupid question. The company I am working with is trying to get ride of most of there IT infrastructure, so that they don't need any more servers. I have been asked to develop a program that takes information from a google spreadsheet and then with this information puts it into a web browser. I am Planning on using Phyton and selenium web driver. Will I be able to install selenium if i host the application as a Google app engine?
The Reason I want/need to use selenium web driver is because I need to put the information from google into a legacy system. The only way to put information in the system is to mimic a user putting the information in manual in a web browser.
Thank you,
Kai
I don't understand what you think Selenium will be doing here. It seems a very strange way to want to get information from one Google property into another.
Google spreadsheets have a perfectly good API that allows you to read the data from your app and display it to users.
Edit after question update Well, now I don't see what you need GAE for. That is for hosting and running websites, and you only seem to want to enter data into an existing website. That's not what it does at all.
I've been looking at this for the past hour and can't find a direct answer. Does this build into a website and anyone who can access a browser have access to it and that's it?
I was curious if it can build for mobile devices like Apple, Android, Windows Phones and also have a web page for it. Pretty much looking for an engine that can build for web/pc + android, apple, windows phones, kindle etc .
Any advice is greatly appreciated, sorry for the newbie question. Thank you in advance.
You can use it to build anything that are served from an URL (e.g. http://example.com/something).
That includes web pages, the "backend" for any application that GET/POST something to an http:// - which means any mobile Apps. And anything else you can imagine that make use of http:// to communicate between a client and a server
And a very important requirement, you need to have some programming knowledge in order to use this. Otherwise you should be looking for more general web hosting service.
How do Mobile Apps communicate with a server? Does this communication happen over HTTP or are there other methods (for instance, I guess a mobile app could open a socket)? Does this differ form device to device? Is there one most commonly used approach? If the communication happens over HTTP how do the URLs look like? Is there a way to identify the app based on the the "User Agent" in the HTTP request if the communication is happening over HTTP?
Mobile Apps can communicate with a server in many ways. Mobile apps can also use other features of the mobile. For instance SMS , MMS. Its pretty open really. Your question could be "how do computers communicate with a server?".
A mobile app can be a simple as loading a web page. This would be http.
The problem with mobile apps. They run on phones. There are lots of different phone manufacturers and its up to them (in some case the mobile phone operators also have weight) what to allow the programmer to do by the choice of operating system.
If you are getting started with mobile apps. Take a look at android its pretty simple to get going with if you know a little about programming.
As for the useragent. It depends but usually if you are using the http api on the phone, you can set the user agent to what ever you want it to be, its just a string.
Hope this helps!
Communication usually happens over http. Applications usually identify themselves in the User Agent field
I'm developing a web based application built on codeigniter which will also have a mobile application(blackberry, iphone, android etc.). The web application will be connecting to a (mysql) database to store/retrieve/update data. I would like to know what would be the best solution for having a mobile application store/retrieve/update data.
Taking into consideration performance, flexibility, Security
Look at your device no different than if it was a browser. The are both clients. All devices (well at least Android and iOS) provide you with ways to make xhr requests, just like your browser does. So you need an API on your server that your device will use to get the data. Note that if your application does not already have an API, you will need to develop one.
I want to implement "mobile website provider" tool. Basically, It intake desktop website url and generate a mobile version of website without edit/add additional content to the desktop website. I am planning to design a product like (what mofuse.com does. This website create mobile websites). This tool should be hassle free to client(who are not tech savvy.)
1.Q) I want to know kind of functionality involved in this process.
2.Q) In Java Technology, I need the tool set and functionality of this actitity.
As I am new to this mobile application environment. Any information(web links) regarding this will be appreciated.
Update:
its not like skweezer.com.
I had a website and I want to "mobile"ize it. 1) What kind of java tool set I have to use to create mobile web site? 2) How to test in multiple mobile devices 3) How the domain name will be created for mobile websites.
Check out our Wapple Architect tech (http://wapple.net) - we allow you to use the same content from web to mobile, but if it's mobile, instead of outputting the html, use a web service to turn it into the right content for that particular handset.
Mobilizes the same domain, no need for m.yourdomain.com or anything like that.
Check http://wapple.net on a mobile for example!
there are certain plugins which mobilise your website .
they check user agent and if user agent is of mobile browser they load another css of mobile and if you want to redirect some another redirect it ther.
you dont have to buy separate domain make sub domain or cname like
m.yourdomain.ccom
wap.yaourdomain.com
try some javacripts code which detect browser type and redirect accordingly.
The best place to learn about mobile toolkits are
WURFL - The open source community built around profiling devices and managing their capabilities (http://wurfl.sourceforge.net/)
Mobile Elements - A tool to help you build mobile sites using a proxy type approach (http://www.mobilelements.com)
Device Atlas - A device profiling tool (http://www.deviceatlas.com)