Scenario :
I am having an application where I am using AWS Lambdas which are written in Kotlin to query data from a relational DB residing in AWS.
--
My problem is that I want to use an ORM for firing these queries. I dont want to use hibernate as it is too heavy and takes too long to setup, and I need a solution that would take up the least time in setting up and firing from the Lambdas. I have looked upon multiple ORMSs like Exposed, Requery, Jooq, Ktorm and Squash.
Is anybody out there having experience with any of these libraries in the serverless context? What are your experiences with them and what would you suggest using in my scenario?
You can have a look at exposed, https://github.com/JetBrains/Exposed
I have been using Squash with the Hikari connection pool for some large projects and I have been very happy with it. I like that is is very extensible and my team has been able to solve any issues that come up, implementing extensions to the dialect and the simplicity of defining TableDefinition classes makes it work well for generating code. It is also very self contained with very few dependencies and light on reflection, so should be good for serverless though I have not personally used it for that.
Squash is less an ORM than an sql abstraction / translation layer that ties into entities and it doesn't try to solve all the problems that something like hibernate does. In my experience ORMs start as simple, efficient, and powerful projects and grow to heavyweight libraries that try to do too much and their complexity begins to cause issues when the developer cannot easily see what's going on in the chain from usage through to the database / storage mechanism.
One negative about squash that deserves mention is that, while it is a jetlbrains official library and created by a kotlin developer, support is limited as orangy, the creator, is quite busy and I have feature pull requests outstanding, with many more of them backed up currently. I chose it because I favored it's simplicity and extensibility among a small but advanced team of developers all capable of improving upon it.
Which ever library you choose I hope these factors assist you in making your decision at the least.
Related
Is there a benefit for using prebuilt connectors to and from SaaS billing platforms like Aria/Zuora when they live between Salesforce and Netsuite used as pure CRM and ERP/Acounting/Finance respectively. That is, versus using an ESB/Integration platform like Mulesoft or Boomi.
We are currently looking at changing billing and ERP systems and having them integrate together and with Salesforce CRM. So the chain would look like:
CRM -- Billing Solution -- ERP
Many of the billing systems have prebuilt connectors that work with ERP systems like Netsuite or Fusion, as well as connectors for Salesforce. Not to mention web service end points/APIs.
But there are integration vendors like Mulesoft and Boomi (basically Enterprise Service Bus PaaS providers) that also allow integration between the services.
I come from a SOA background and tend to favour a standalone ESB to connect the systems but due to my lack of familiarity with Saas ERP systems don't understand the benefits and pitfalls in the prebuilt connector vs ESB debate. I understand the concepts behind avoiding point to point integration, which would turn around to being a benefit for using ESBs. But is there a benefit for using prebuilt connectors within the SaaS platforms ... and are there serious downsides (my main concern).
Can anyone provide some insight here? I am not asking for "which one is best", just some real world experience good or bad that could help someone make these kinds of decisions.
I cannot provide a comprehensive comparison between the services you plan on using, but your question is quite interesting so I thought I'd share my thoughts and experience and hope you'll benefit from it.
Prebuilt connectors are not something new - they existed long before SaaS and iPaaS became a thing. So their pros and cons are still the same, the main issues you will be looking at are still very much related to the lack of flexibility you'll be facing and of course, the shortcomings of point-to-point integration. Things are somewhat refracted via the prism of SaaS/iPaaS but I believe that most aspects are still relevant.
Prebuilt Connector capabilities and support
You need to assess to what extent a prebuilt connector really covers the integration between the two systems. Services like salesforce take pride in their customizability and extensibility by using 3rd party extensions. In most cases the connector will be following a one-size-fits-all approach that only satisfies the most common and simple of all integration needs. It's all fun and games until something has to change. It is not possible to know in advance what you could need the future but think about it - would you be able to count on having your customisations and extensions covered by the prebuilt connector in case you decide to integrate them as well?
Another point you must consider is support - what happens if one of those companies decide to suddenly announce that they will stop supporting future integration via prebuilt connectors you are already using? You should check to see if there are any guarantees for you.
Tight Coupling and Service provider lock-in
Using point to point connectors will couple systems to each other so you’ll be severely limiting your options to switch between platforms if you need to at some point. It might seem a fairly simple integration scenario now, but adding more systems to the mix over time generally makes things even worse, since you are going to have dependencies here and there, and not every new system will have a connector out of the box to integrate easily with all the others you're already using. Having a middleware gives you the precious ability to map and transform data if needed, and maybe even apply some business logic that makes your life much easier (and cheaper). Also you'd be able to replace a system without having to replace others depending on it.
Consider your scenario: if you decide to change the billing system, you will have to find one that’s being properly supported by both the CRM and ERP providers. Thus, you could potentially remain locked into using exactly these three, even though for example they don’t fit your needs anymore or there is something else on the market that would have given you great competitive advantage if only you could integrate with it.
Orchestration and future investments
An important note about the p-2-p scenario is that you will not be able to implement process services that span across all the systems if needed. The added flexibility and benefits of using even simple forms of orchestration (I’m not even talking about achieving what can be achieved with a full featured business process management) will be off reach for your business. When the market changes and Time to Market is the deciding factor you may be not be prepared.
Thoughts on choosing iPaaS
Using iPaaS platform looks like a much better decision in the long run. Yet, you still have to make sure that the platform does not just give you some set of predefined connectors and drag&drop beauties (they all do), but also the ability to easily implement your very own integrations from scratch while supporting industry standards. I think that it is absolutely crucial to have this kind of flexibility when talking about an ESB solution, be it in the cloud or on premises.
The potential cons of the iPaaS approach would be:
you come to depend on yet another service provider and you will have more costs because the service is not free;
your data travels to another service provider, so there is additional risk in terms of security, no matter what the service providers may try to tell you;
more upfront effort spent on design and implementation;
additional burden, related to having to maintain integration and accommodate potential changes (however rare they might be) if a new version comes out.
Conclusion
It’s all really a tradeoff between desired flexibility and the investment that you’re willing to make. Your decision will heavily depend on the current state of your business and your growth expectations going forward, rather than the purely technical side of things.
I hope my thoughts gave your some perspective. Please update the question with your decision and reasoning when the time comes. Good luck!
I am looking to dip my hands into the world of Multi-Model DBMS, I have no particular use cases, just want to start learning.
I find that there are two prominent ones - OrientDB vs ArangoDB, but was unable to find any meaningful comparison, unopinionated between them. Can someone shed some light on the difference in features between the two, and any caveats in using one over the other? If I learn one would I be able to easily transition to the other?
(I tagged FoundationDB as well, but it is proprietary and I probably won't consider it)
This question asks for a general comparison between OrientDB vs ArangoDB for someone looking to learn about Multi-model DBMS, and not an opinionated answer about which is better.
Disclaimer: I would no longer recommend OrientDB, see my comments below.
I can provide a slightly less biased opinion, having used both ArangoDB and OrientDB. It's still biased as I'm the author of OrientDB's node.js driver - oriento but I don't have a vested interest in either company or product, I've just necessarily used OrientDB more.
ArangoDB and OrientDB are both targeting a similar market and have a lot of similarities:
Both are multi-model, you can use them to store documents, graphs and simple key / values.
Both have support for Gremlin, but it's firmly a second class citizen compared to their own preferred query languages.
Both support server-side "stored procedures" in JavaScript. In both systems this comes via a slightly less than idiomatic JavaScript API, although ArangoDB's is a lot better. This is getting fixed in a forthcoming version of OrientDB.
Both offer REST APIs, both aim to be usable as an "API Server" via JavaScript request handlers. This is a lot more practical in ArangoDB than OrientDB.
Both are distributed under a permissive license.
Both are ACID and have transaction support, but in both the transactions are server-side operations - they're more like atomic batches of commands rather than the kinds of transactions you might be used to in a traditional RDBMS.
However, there are a lot of differences:
ArangoDB has no concept of "links", which are a very useful feature in OrientDB. They allow unidirectional relationships (just like a hyperlink on the web), without the overhead of edges.
ArangoDB is written in C++ (and JavaScript), whereas OrientDB is written in Java. Both have their advantages:
Being written in C++ means ArangoDB uses V8, the same high performance JavaScript engine that powers node.js and Google Chrome. Whereas being written in Java means OrientDB uses Nashorn, which is still fast but not the fastest. This means that ArangoDB can offer a greater level of compatibility with the node.js ecosystem compared to OrientDB.
Being written in Java means that OrientDB runs on more platforms, including e.g. Raspberry PI. It also means that OrientDB can leverage a lot of other technologies written in Java, e.g. OrientDB has superb full text / geospatial search support via Lucene, which is not available to ArangoDB.
OrientDB uses a dialect of SQL as its query language, whereas ArangoDB uses its own custom language called AQL. In theory, AQL is better because it's designed explicitly for the problem, in practise though it feels quite similar to SQL but with different keywords, and is yet another language to learn while OrientDB's implementation feels a lot more comfortable if you're used to SQL. SQL is declarative whereas AQL is imperative - YMMV here.
ArangoDB is a "mostly-memory" database, it works best when most of your data fits in RAM. This may or may not be suitable for your needs. OrientDB doesn't have this restriction (but also loves RAM).
OrientDB is fully object oriented - it supports classes with properties and inheritance. This is exceptionally useful because it means that your database structure can map 1-1 to your application structure, with no need for ugly hacks like ActiveRecord. ArangoDB supports something fairly similar via models in Foxx, but it's more like an optional addon rather than a core part of how the database works.
ArangoDB offers a lot of flexibility via Foxx, but it has not been designed by people with strong server-side JS backgrounds and reinvents the wheel a lot of the time. Rather than leveraging frameworks like express for their request handling, they created their own clone of Sinatra, which of course makes it almost the same as express (express is also a Sinatra clone), but subtly different, and means that none of express's middleware or plugins can be reused. Similarly, they embed V8, but not libuv, which means they do not offer the same non blocking APIs as node.js and therefore users cannot be sure about whether a given npm module will work there. This means that non trivial applications cannot use ArangoDB as a replacement for the backend, which negates a lot of the potential usefulness of Foxx.
OrientDB supports first class property level and database level indices. You can query and insert into specific indexes directly for maximum efficiency. I've not seen support for this in ArangoDB.
OrientDB is the more established option, with many high profile users. ArangoDB is newer, less well known, but growing fast.
ArangoDB's documentation is excellent, and they offer official drivers for many different programming languages. OrientDB's documentation is not quite as good, and while there are drivers for most platforms, they're community powered and therefore not always kept up to date with bleeding edge OrientDB features.
If you're using Java (or a Java bridge), you can embed OrientDB directly within your application, as a library. This use case is not possible in ArangoDB.
OrientDB has the concept of users and roles, as well as Record Level Security. This may be a killer feature for you, it is for me. It also supports token based authentication, so it's possible to use OrientDB as your primary means of authorizing/authenticating users. OrientDB also has LDAP integration. In contrast, ArangoDB support only a very simple auth option.
Both systems have their own advantages, so choosing between them comes down to your own situation:
If you're building a small application, and you're a web developer optimizing for developer productivity, it will probably be easier to get up and running quickly with ArangoDB.
If you're building a larger application, which could potentially store many gigabytes or terabytes of data, or have many thousands of concurrent users, or have "enterprise" use cases, or need fine grained security controls, OrientDB is the one for you.
If you're storing RDF or similarly structured linked data, choose OrientDB.
If you're using Java, just choose OrientDB.
Note: This is (my opinion of) the state of play today, things change quickly and I would not underestimate the ruthless efficiency of the awesome team behind ArangoDB, I just think that it's not quite there yet :)
Charles Pick (codemix.com)
My project is a data manager library which among other things feature a parser of a DSL that generates SQL queries that are semantically given an SQL database schema, using SQLite as for storage. Parts of the testing code then need the database to be set up (scaffolding) to provide the specific scenario in the database within which I need to test for the function in terms of assertions on output.
Are there testing frameworks for Haskell that are better or worse for this, you think? I thought I'd ask for advise first before I spend too much time learning the different frameworks out there.
Personally, I think the major testing frameworks in Haskell are equally suited for this. For persistent, I use hspec, and have no complaints. Many people also use tasty, and that seems to work out well too. I honestly can't give you a list of advantages and disadvantages between the two, I'd recommend looking into them and making a decision.
The other two players I'm aware of are HTF and test-framework. I don't know anything about HTF, but AFAIK, test-framework is essentially superseded by tasty.
I believe all of these frameworks work well with both HUnit and QuickCheck.
I know this is not a programming question per se, but I wanted to get as much input from the SO community on a new project I hope to get started. The project is from being started from scratch and thus every decision for programming languages, databases, frameworks, platforms and what not are up in the air. I'm hoping to get your opinion on the matter, what you feel are the strengths and weaknesses of each option.
Database:
Currently I have the option of using MSSQL or MySQL. While I am leaning towards using MySQL because it is free and most probably has all the features I need. However, there is the possibility of having a lot of hierarchical data and the new hierarchical data type in MSSQL is quite appealing. Does it really simplify matters that much? Also MSSQL supports many more advanced SQL functions that may or may not be useful in the long run. While for development I can get access to Server 2008, multiple licenses as the development team grows and for production, are the costs justified?
Programming Languages:
The project will have a web based front end UI and a server based component that will do some heavy lifting.
For the web based UI, I was thinking of maybe doing Apache/IIS with PHP or IIS with ASP.Net in C#. I'd like to use a good framework to properly utilize good design patterns that should structure the code and development of the app. As well as make modifications in the long run easy to implement. I also want the GUI to look good and don't like the idea of buying .Net controls from component vendors. Instead I prefer the idea of using good CSS, and open sources like YUI and javascript to make the UI sleek.
For the server based component, I was thinking of using C#. I have no real development experience in C++ and I'd like good libraries and sufficient speed is good enough. However, while the web based UI and server based component is loosely coupled, there may be instances where the UI needs to communicate (call methods and what not) with the server based component and I want to pick languages/frameworks that will play nice with each other.
All suggestions on frameworks to incorporate are welcome.
Version Control:
I have had good experiences with SVN and a pretty bad experiences with TFS. I've never worked with GIT. Which do you think is better in terms of features as well as general developer familiarity. I want to pick something that other developers will know and not have trouble with.
I apologize if the questions are bit redundant or I'm not providing enough information or using bad terminology. I plan to edit and improve the question as I get feedback. Thanks!
EDIT:
Who: This would most probably be a startup formed of college students or junior developers. I want the project to utilize technologies that most people are familiar with or are easy to pick up.
What: I'd need hours and days to explain the solution. But in the end when you break it down, its a web based UI (think standard web app to just manage database data) that would be used to knowledgeable clients. The server based component would be very separate except for the fact that it should be able to communicate with the web app.
I can provide more information as required but I would appreciate an opportunity for users to answer and provide their ideas before you hastily close the question.
Obviously it depends a lot on specific requirements, but then again, even with those I probably wouldn't be able to tell for sure!
I've been working on a from-scratch project myself for a couple of months, and have generally found:
Choosing Microsoft for all the layers just goes down much easier (my subjective opinion). For example I would use C# for the UI, the back end, and use MSSQL for the database. Nothing at all wrong with non-Microsoft vendors, I'm no Microsoft fan-boy, I just struggle to get productive with unfamiliar tools. Depends where your experience lies though.
Database: In particular I've found that .NET and MSSQL go easily together. When I started the project I was using a PostgreSQL (because it's free, fully featured and has open-source warm fuzzies). However I abandoned it in favour of MSSQL simply because it was taking me too long to get database work done in an unfamiliar language with unfamiliar tools. Also, I'm not sure MSSQL is so expensive anymore, for example for a web application, MSSQL 2008 Web Edition is pretty damn cheap per-processor I think (only on SPLA licensing though). If you're concerned about database features in a free implementation though, personally I think PostgreSQL has a very full feature set, nicely standardised, and rapidly growing.
UI: I'm pretty inexperienced, but ASP.NET MVC looks far less painful to me than ASP.NET Web Forms. I like PHP too, but again I'd match the UI language with the back-end language, so would recommend .NET.
On frameworks, I'm immersed in DALs at the moment. I like Subsonic for lightweight data, NHibernate for heavy-weight.
I still have a long way to go with my project so perhaps I can only see the short-term benefits and drawbacks at the moment. But in general I would say: use the technologies that you're most comfortable using, as you'll be way more productive and the end result will probably be about the same anyway. If you want to learn new technologies though, and who doesn't? - go ahead, just expect it to take a lot longer.
Didn't want to answer 'cause it's so open ended. But a few points:
Money
First, check out BizSpark. That should take care of any money aspect for 3 years. For a service company, that means not only free VS Team Suite and Office and so on, but free Windows, SQL, etc. If your startup can't afford to spend a bit on MS tech in 3 years, it's probably a bad business. So that takes out licensing.
On a similar note, Sun has Startup Essentials. Could be interesting on the hardware side of things, but I haven't actually competitively priced them versus Dell/HP.
Software
It doesn't sound like you have hard enough requirements to say "oh, this slightly-less-popular software X is perfect for my domain Y and is gonna give me a very big boost". In fact, your project might not be like that at all. Maybe it, technically, is going to be a relatively plain application just pushing data around or whatever. You didn't specify.
For a small startup, personal productivity is probably going to trump any other argument. If your people are excellent in X, then that's one of your top arguments right there.
If you really don't have any particular system you're most comfortable with, be conservative. Stick with .NET or Java, as they'll give you the widest range of useful possibilities.
As far as things like OS and Database, I'm biased, but I think Microsoft will give you platforms that are easier to take advantage of than you'll find elsewhere. For instance, setting up load balancing, clustering, centralized authentication, managing servers (updates, events, etc.) is going to be easier to get going on Windows than it would be on another platform, assuming you're not an expert in either. Configuring SQL Server, even the advanced features, is a piece of cake. (Go time someone who knows neither: Setup a DB mirror in MSSQL and MySQL -- which is going to take more work?) Again, this is all predicated on you not having experts in a particular set of technology.
Don't mix -- whatever you do, stick with the platform. If you go .NET, MSSQL is going to work better with the data providers (or things like Linq-to-SQL). If you decide to do PHP, then use MySQL as everyone else uses it and you'll encounter less resistance. If you're not inventing stuff on the technical side, don't become an edge case.
You should pick the platform first, then the language that is best for that platform (if there is any choice).
One thing you should consider is the labor pool, and labor pool cost, for specific platforms and languages. Human Resources can often get cost metrics, if you don't have ideas already.
In my town, for example, .NET platform is much more expensive per Software Engineer than open source, because the .NET developers have a higher rate (40% roughly). C# is a little higher rate than VB.NET, but also tends to bring more well rounded candidates.
Just to throw in something totally different: How about weblocks as a web framework? It uses Hunchentoot as a server, which can run either standalone or with Apache. This is all done in Common Lisp. Weblocks can use cl-sql as a backend store, which can connect to many different RDBMs (MySQL, PostgreSQL, Oracle, ODBC, SQLite).
I am using C# 3.0 / .NET 3.5 and planning to build an eCommerce website.
I've seen NHibernate, LLBLGEN, Genome, Linq to SQL, Entity Framework, SubSonic, etc.
I don't want to code everything by hand. If there is some specific bottleneck I'll manage to optimize the database/code.
Which ORM would be best? There is so much available those day that I don't even know where to start.
Which feature(s) should I be using?
Links, Screencast and Documentation are welcome.
I've been using nHibernate which is a very good free solution. The one downside is the lack of documentation, which causes a slightly steep rampup time. But once you get the basics down it really speeds up development.
I like Fluent nHibernate for a way to configure without the xml files. The one thing I suggest though is to abstract out your data access from your application. this way should you choose wrong you don't have to worry about re-coding the App tiers.
I can only really speak for LINQ-SQL and can say that it is:
Easy to use
Quick to get you up and running
Good for simple schemas and object models
but it starts to fall down if:
You're using a disconnected (tiered) architecture because its datacontexts require the same object instances to perform tracking and concurrency (though there are ways around this).
You have a complex object model / database
Plus it has some other niggles and strange behaviour
I'm looking to try EF next myself and MS seem to be quietly dropping LINQ-SQL in favour of EF, which isn't exactly a ringing recommendation of LINQ-SQL :)
That depends on the architecture of the data model. I can speak to the effectiveness of SubSonic, since I'm in the process of launching a web app that it backs.
I've run into problems with JOINs and DISTINCTs while using SubSonic. Both times, all I had to do is patch the source and rebuild the DLL. Now, I'm not at all averse to something like this, but you might be.
Other than those two problems, SubSonic is a joy to use. Selects are very easy and flowing. It maps fairly closely to SQL, much the same way LINQ does. Also, SubSonic comes with the scaffolding function that should be able to pre-build certain pages for you. I'm not sure how effective it is, since I like to do that stuff myself.
One more thing, selection of specific rows as opposed to * is slow, but only in debug mode. Once you compile for release, it's actually faster.
That's my two cents.
I started out using Linq to SQL as the whole linq integration is awesome, but if you want to do Model First rather than Schema First and you want to have a rich domain model then nHibernate\Fluent nHibernate is really the way to go. We switched to this and is far simpler, better supported than l2s. However for straight dragging your schema into the dbml code generator, linq to sql is great.
I have also heard very good things about Mindscape Lightspeed but have not used it.