I am working on a project where I need to run Hadoop Over Mobile devices. I have searched "Mobile Hadoop" but I was not able to find any distribution of Hadoop which runs over mobile devices. Is there any distribution of Hadoop which runs over Mobile and if not, How can I write my own Hadoop which can do it.
To run Hadoop on Mobile devices seems a joke. Typically, Hadoop runs on commodity hardware. What do you exactly mean by Mobile devices? If you mean mobile phone, may there not exist Hadoop distributions for your project. Even though you can deploy Hadoop on a Mobile devices cluster, what can that cluster do?
Related
When developing a DApp, one has to put part of the tech on centralized services such as AWS or Google Cloud.
I am wondering if there are any other projects that allow to decentralize database and computational power in order to have fully decentralized and unstoppable DApps?
You might want to check Tea Project since it offers;
decentralized cloud
rich and fully decentralized dapps
trusted computation environment
For instance, IPFS offers decentralized storage but lacks a decentralized compute layer to go with it. Projects like Helium decentralize data transmission but are missing a compute layer to directly run dApps on network data. Tea Project solves the current issues of decentralized cloud ecosystem. You can check the website and the infographics attached below.
Tea Project Infographics
Some decentralized solutions are already emerging in order to decentralize databases and computational power.
For instance, Aleph.im is a scalability cross-chain project that does exactly that: decentralized database and computation; compatible with Substrate/Polkadot, Ethereum, BinanceChain, and more.
Check out its SDK in Javascript and Python.
Will graphlab, apache mahout or lenskit work on google app engine ?. If it won't, how will i be able to use collaborative filter on gae ?
There should be no problem in principal running LensKit on Google AppEngine, but doing the model build will be tricky (you'll need a way to run that job).
Apache Mahout integrates nicely with platforms such as Hadoop for distributed processing of the data. However, most of the algorithms are designed to work standalone as well. You should be able to run them within App Engine without an issue. For scalability though, I would consider switching to Hadoop.
I work for a big company - we are not to big on "open" technologies. Our security people are so paranoid that we cannot even login to most web-services (including Google!).
Us devs really like Google technologies, in particular the App engine. Given that we cannot host company services outside the company infrastructure can we do the opposite? I'd like to use some of our department's servers to make a small GAE-compatible grid and use them to run my own application.
We do not need the whole of the GAE experience, for example we do not need Google's APIs - I just want to use the Google BigTable technology for our private projects.
Can this be done?
FYI, We have about 10 servers available for this project (they do not have to all be used). And to complicate matters, most of our machines run Windows.
Thanks
AppScale http://github.com/AppScale/appscale
AppScale is an open-source hybrid cloud platform. AppScale implements
a number of popular APIs including those of Google App Engine,
MapReduce (via Hadoop), MPI and others. AppScale executes as a guest
virtual machine (guestVM) over any virtualization layer that can host
an Ubuntu Lucid image.
Typhoon App Engine http://code.google.com/p/typhoonae/
The TyphoonAE project aims at providing a full-featured and productive
serving environment to run Google App Engine (Python) applications. It
delivers the parts for building your own scalable App Engine while
staying compatible with Google's API.
There is the the open source project AppScale which mimics the App Engine framework.
It being in development for quite some time and can be hosted on a private cloud.
Check out CapeDwarf (http://www.jboss.org/capedwarf):
JBoss CapeDwarf is an implementation of the Google App Engine API,
which allows applications to be deployed on JBoss Application Servers
without modification. Behind the scenes, CapeDwarf uses existing JBoss
APIs such as Infinispan, JGroups, PicketLink, HornetQ and others.
There are to popular clones of the Google's BigTable: HBase and Cassandra. Both implement the same concept but are built completely different internally. Selection between them depends on Your requirements for the consistency and high availability.
I'm pretty new to Appscale and GAE. Say I have a web application on GAE (offers hassle-free maintenance on server security etc), if I move the application to a Xen VPS host using Appscale, do I still enjoy hassle-free maintenance (since Appscale is based on GAE)?
No. GAE is two things - a platform, and a service. When you run your apps on Google's GAE infrastructure, Google is providing the service of keeping the infrastructure up and running. In exchange, you pay them (only if you use more than the free quota.) If you choose to install Appscale on your own hosts (doesn't matter if they are virtual or not), you will now have the job of keeping things running. In exchange, you save the money you would have been paying to Google.
This is no different than if you had a single HTML page you wanted to serve - you could pay a web hosting company to host it for you, hassle free, or you could install IIS or apache on your own computer, and serve it yourself.
may i know how portability is gae application. let say i have an
application that using comet feature, task queue..etc and not using
bigtable and i want to deploy on vmware/amazon will it run?
No. Google App Engine and Amazon Cloud Services have different APIs.
Many of the concepts are similar; it would certainly be possible to port many applications between the two and achieve similar functionality - but you'd have to modify a lot of code. If you have a lot of data stored in one system, you'd have to write custom code to export it from that system and import it into the other, too.
AppScale is designed for this kind of portability.
AppScale is a platform that allows
users to deploy and host their own
Google App Engine applications. It
executes automatically over Amazon EC2
and Eucalyptus as well as Xen and KVM.
It has been developed and is
maintained by the RACELab at UC Santa
Barbara. It supports both the python
and java Google App Engine platforms.
Can't say for sure if it's something that will work for you but it's probably worth looking into.