I am working on an end-of-year project and I would like to investigate the benefit of using a ledger database or immutable against using a full blockchain. In my theory for a certain application, it would be that the ledger database is simpler to use more than a blockchain. I tried to look for some open source ledger database but I can only see 2 options in the search Amazon QLDB and immudb. Maybe I am missing some project? Is the QLDB using immudb as many of the AWS services are using open source?
Amazon QLDB and immudb are independent projects even though the technology behind both projects is similar (e.g. both use Merkle Trees verify the history of records).
The main difference is that immudb is and open-source project released under the Apache-2.0 License which means that you can easily install it in your own environment.
Related
I am trying to decide which add-on DB to use with my application when I deploy it on AppHarbor. I've two choices: JustOneDB or Cloudant. I am planning to develop a web and mobile application, which will should work with Terabytes of data.
I am searching for the easiest solution to deploy my database, without me needing to partition the DB and the tables. I want a DB that can handle a very large amount of data, but takes the sharding and partitioning architecture building away from the developer.
I also want a solution that will allow me to easily backup my large database and easily restore it.
From what I've read, Cloudant and JustOneDB are the two most popular ones, and those are available as add-ons on AppHarbor for easy deployment.
I need your recommendations on which one I should go with, the cons and pros of each one. I am developing my application in ASP.NET and C# inside Visual Studio.
There's a recent post on the Cloudant blog about using the MyCouch .Net library with Cloudant databases:
https://cloudant.com/blog/how-to-customize-quorum-with-cloudant-using-mycouch/
Cloudant also offers free hosting up to a greater than $5 bill and can work with Apache CouchDB's replication if you want to develop locally and sync it to the cloud for production/deployment. Multi-master replication isn't something many other databases offer.
Best of luck with your application!
MyCouch.Cloudant was just released. Except from CouchDb and Cloudant core feature support the MyCouch.Cloudant NuGet package adds support for Searches. There will be more Cloudant specific features added to this. It's written in C# and supports .Net40, .Net45 and Windows store apps.
You will find more info about MyCouch in the GitHub repo.
You should probably also consider MongoDB and RavenDB.
If you're just starting out, your first concern should probably be to find a database that'll let you quickly get started and build the application you have in mind. When the application becomes a success and actually attracts terabytes of data, you can start worrying about how to scale it. If the application is soundly architected, adapting it to use an appropriate datastore should not be a monumental task.
Comment removed by originator.
Requirements for archival type software
1. Data/Image/possibly video.... upload/search/retrevial/edit from web.
2. Easily implemented user defined Custom Fields
3. Easy backup.
4. Low cost ... either opensource or very low cost
I am a very novice programmer. My primary goal is to manage a collection and publish it to the web.
Options
A. Open source software such as collective access
Problems: Custom fields not supported. Continued support? Portablity of
database?
B. Use Microsoft Access and then use MVC or other development platforms to eventually
publish to the web.
Problems:Difficult to integrate to web?
C. Design my own MVC database application.
Problems:Difficult for novice programmer? Custom Fields and Upload of various data
formats difficult to implement?
Sounds like you are looking for a Digital Assets Management system. I found ResourceSpace (http://www.resourcespace.org/) and Razuna (http://www.razuna.org/) very useful for similar projects - both fall into your A category.
Requirements for archival type
software 1. Data/Image/possibly
video.... upload/search/retrevial/edit
from web. 2. Easily implemented user
defined Custom Fields 3. Easy backup.
4. Low cost ... either opensource or very low cost
Hi there,
As mentioned here before, but Razuna will satisfy your requirements quite well.
It can manage images, documents, videos and audios. It will share folderd and collections on the web with access permissions and will allow you to search among the different kind of assets as well.
Moreover, it can handle metadata of all this asset. It will not only read metadata, but also WRITE metadata, also. Furthermore, you can set the custom fields for each asset type and users will have a web interface to work with.
Razuna supports different databases (H2, MySQL, MS SQL and Oracle (soon DB2)) and let's you migrate from one db to another with ease (backup / restore option).
Best of it all: It is available under a open source license for you to deploy and enjoy today. You can get it at http://razuna.org.
Kind Regards,
Nitai
PS: I'm the main developer and founder of Razuna.
I am almost done developing a Django project (with a few pluggable apps).
I want to offer this project as a SaaS (something like 37signals.com).
i.e: customer1.product1.com , customer2.product2.com etc
product1 could be the basecamp
product2 could be highrise
and so on.
I want to know how the project should be structured for these products.
Should there be a single project under which all products will be an application.
---- OR ----
Should I be making different projects for all the products.
Also interms of database.. should all the products look into a single database or we should have seperate databases for each product.
I am looking out for the most efficient and scalable way to do this.
Thanks
If you want the apps to interact in any way (e,g, share user accounts), they need to be on the same database. Until Alex Gaynor's great work on multiple database support gets merged into trunk, Django doesn't support multiple databases.
Along the same lines, Django's sites framework may be of interest to you. Without any more information about your intentions, it's difficult to give any better advice.
If you have a few big clients, giving each their own database and Django instance might make sense. Especially if you need to do customisations for them.
For a SaaS app where there are potentially lots of clients with "create your own" setup and self-service admin, you probably want one database and add subdomain support in the Django app.
Here's some good tips and code on how we did it: Handling Subdomains in Django.
There's an open source project claiming to handle a lot of the infrastructure for a basic 37Signals-like Saas site: http://github.com/saas-kit/django-saas-kit
We're writing an open source tool that is designed to work against multiple database products (really, it's designed to work with any database that has a JDBC driver available). However, because it does DDL (not DML), we need to test against different products.
While MySQL, PostgreSQL and other open source db's are easy for us to test on (since we - the developers - all use these daily), testing those products is easy. But, finding different installations of other, commercial database products is a different story.
Does anyone know of a "database sandbox", or similar service that would allow us to test code against multiple database products without having to buy and install them?
Thanks in advance!
I don't know of such a service but it may well be worthwhile contacting some of commercial vendors as it is often in their interests to have interoperability with open source libraries. The other way to approach it would be to find a company that would like to use your tool with one of the commercial db's (because they are already tied to that vendor in-house) and ask them if you can get access or spend some time in their office. This is an approach I used when developing a Continuous Integration tool which needed to work with different commercial source control systems and it worked surprisingly well.
You may also want to get in touch with open source orgs like Codehaus or Apache as they have established open-source links with commercial vendors and may be able to give you access to the DBs you need.
You may want to use an ORM, such as Hibernate, or some other open-source ORM, as those will be tested on different databases, and you can then develop knowing that all you have to do is unit test the DAO, but not be so concerned about the different databases.
I was looking into the possibility of using CouchDB. I heard that it was similar to Lotus Notes which everyone loves to hate. Is this true?
Development of Lotus Notes began over 20 years ago, with version 1 released in 1989. It was developed by Ray Ozzie, currently Chief Software Architect for Microsoft.
Lotus Notes (the client) and Domino (the server) have been around for a long time and are mature well featured products. It has:
A full client server stack with rapid application design and deployment of document oriented databases.
A full public key infrastructure for security and encryption.
A robust replication model and active active clustering across heterogeneous platforms (someone once showed a domino cluster with an xbox and a huge AIX server).
A built in native directory for managing users that can also be accessed over LDAP.
A built in native mail system that can scale to manage millions of users with multi GB mail files, with live server access or replicated locally for off-line access. This can interface with standard internet mail through SMTP and also has POP and IMAP access built in. The mail infrastructure is a core feature that is available to all applications built on Notes Domino (any document in a database can be mailed to any other database with a simple doc.send() command).
A built in HTTP stack that allows server hosted databases to be accessed over the web.
A host of integration options for accessing, transferring and interoperating with RDBMS and ERP systems, with a closely coupled integration with DB2 available allowing Notes databases to be backed by a relational store where desired.
Backwards compatibility has always been a strong feature of Notes Domino and it is not uncommon to find databases that were developed for version 3 running flawlessly in the most up to date versions. IBM puts a huge amount of effort into this and it has a large bearing on how the product currently operates.
-
CouchDB was created by Damien Katz, starting development in 2004. He had previously worked for IBM on Notes Domino, developing templates and eventually completely rewriting one of the core features, the formula engine, for ND6.
CouchDB shares a basic concept of a document oriented database with views that Notes Domino has.
In this model "documents" are just arbitrary collections of values that are stored some how. In CouchDB the documents are JSON objects of arbitrary complexity. In Notes the values are simple name value pairs, where the values can be strings, numbers, dates or arrays of those.
Views are indexes of the documents in the database, displaying certain value, calculating others and excluding undesired docs. Once the index is build they are incrementally updated when any document in the database changes (created updated or deleted).
In CouchDB views are build by running a mapping function on each document in the database. The mapping function calls an emit method with a JSON object for every index entry it wants to create for the given document. This JSON object can be arbitrarily complex. CouchDB can then run a second reducing function on the mapped index of the view.
In Notes Domino views are built by running a select function (written in Notes Domino formula language) on each document in the database. The select function simply defines if the document should be in the view or not. Notes Domino view design also defines a number of columns for the view. Each column has a formula that is run against the selected document to determine the value for that column.
CouchDB is able to produce much more sophisticated view indexes than Notes Domino can.
CouchDB also has a replication system.
-
Summary ( TL;DR ) : CouchDB is brand new software that is developing a core that has a similar conceptual but far more sophisticated design to that used in Lotus Notes Domino. Lotus Notes Domino is a mature fully featured product that is capable of being deployed today. CouchDB is starting from scratch, building a solid foundation for future feature development. Lotus Notes Domino is continuing to develop new features, but is doing so on a 20 year old platform that strives to maintain backwards compatibility. There are features in Notes Domino that you might wish were in CouchDB, but there are also features in Notes Domino that are anachronistic in today's world.
It is the Notes application and UI that people usually hates. Not the architecture behind.
Damien Katz worked at Iris (Lotus), but he was not the guy behind the Notes Database. He is well-known in the Lotus Notes community for redesigning the Notes Formula Engine.
There are definitely some similarities between CouchDB and Lotus Notes, such as their document-oriented, non-relational data, and replication capabilities, but they are more disparate than similar. CouchDB is a database server and Lotus Notes is an enterprise-level collaboration platform.
#Lex, You should prehaps say what version of Notes/Domino you are working on because your comments are incorrect.
"No transaction support" - Domino has transactional logging. If you want more complex transaction logging that is also available within coding.
"not well suited for handling multiple data transactions" - Actually it handles them just fine. You have document locking and replication conflict resolution. Depends a lot on how you set up your application to handle workflow.
"No separation between production/dev environments." - False. The only way this could be true is if you had a badly deployed environment. Developers normally should have 0 access to deploy design changes to the production environment. They would work off a template which does not replicate to main servers. Once updates are done and approved then the administrator deploys it. They do this by taking the template and signing it with a controlled signature allowed to run on production, then drop the template in and update the design of the related applications.
"The more data lotus notes contains, the more views will likely get created" - This comment makes absolutly no sense what-so-ever. I don't believe you have used Notes/Domino in any professional ability.
"lotus script is not object oriented" - Yes you make good points there. However it doesn't mean that the language is flawed. Also they have made a large number of improvements since 8.x and with 8.5.1. For example built in web services support (point to WSDL and LS code is made for you). 8.5.1 Also has a lot of new designer features like Code Templates, auto-completion, LSDoc popup help on your own functions, etc.
You also only touch on LotusScript. Yet you can also code in:
Java, SSJS/DOJO (XPages), Javascript, #Formula language, Web Services (SOAP/REST), C-API, Eclipse Plugins(RCP). Output in JSON as well as XML.
8.5.1 Designer client is now free to download if you want to test it out.
So while I believe I am not in a position to comment on CouchDb you most certainly are not on Notes/Domino.
Lotus Notes client/Domino server is comprised of an object("document")-storage (not relational) mechanism, has fully integrated certificate-based security model / user management and conflict-resolution for syncing offline/online changes to data - it's a platform for distributed applications.
"CouchDB is a document-oriented, Non-Relational Database Management Server (NRDBMS)."
CouchDB is accessible via a REST style API.
There's a podcast interview with Jan Lehnardt of the CouchDB team here.
Without going back and listening to it again, I believe that Damien Katz, who was the initiator and is still the lead developer on CouchDB was also the guy behind the Notes database. So there's a sense in which CouchDB is a better Notes DB, I guess. He explains some of the differences in his blog.
It's similar to how Notes deals with data in that everything is a document of arbitrary structure, and you have views over those documents instead of tables and records like you'd have in a relational database. The replication etc also has some similarities.
There isn't anything wrong with the Notes server architecture, people don't hate that so much. It's more the implementation and bloat that comes with Notes.
CouchDB has no front end either, just a server component. The Notes client sucks, and that is what people REALLY hate. Have you ever tried to email uh I mean "memo" something from Notes? Not pleasant :(
Comparing Apples & Oranges
Lotus Notes Domino hasn't changed much and there is not a NoSQL service option on-prem or cloud for Notes Domino v12 or any earlier version. Domino is not cloud based tech.
When it comes to NoSQL, Domino uses NoSQL for its own application solutions built in Domino. There was an attempt with Domino Access Services which is based on Java 6, Rest API still uses Vectors in v12. This service is ok, not robust, it provided a way to interface with data in a NSF. Remember, Domino is key value pairs storage and very slow on large data sets because of the security model, each document is checked for readers and authors with every search to identify if the document can be viewed by the user. Domino is still Web 1.0.
With CouchDB one can build app on mobile and deploy it. There is no way to do the same with Notes/Domino because of the Domino Server. Domino dev also only supports MS Windows and the IDE is based on older versions of Eclipse, to this day v12, there is no way to use dual monitors with the Domino IDE. Ask any Domino Developer, they hate being forced to use a IDE on a specific platform that cannot keep up with industry.
Couch has gone through many changes as well, brief history:
CouchDB started by Damian Katz, IBM Lotus Domino engineer
Apache project BigCouch is born ; scalability and clustering added
Cloudant is born ; BigData and IBM funding and IBM Cloud offering
CouchDB 2.0 is born; Cloudant + BigData merged back into CouchDB
CouchDB 3.0 is born; Enhanced security and prep for Foundation DB
CouchDB 4.0 is born; architecture changed to Apples Foundation DB
https://www.dataengineeringpodcast.com/couchdb-document-database-episode-124/