We want to implement an API, we have a database located on a central server, and a network of many computers.
On these computers, several local programs will be developed in the future using different programming languages, some in java, some in perl, C++, ... etc.
These local programs should be able to access the API functions and interact with the database.
So in what language should the API be written ? so that it would be binding to the other languages. Is there any specific architecture that should be implemented ?
Is there any link that would provide useful information about this ?
If the API is pure database access, then a REST web service is a reasonable choice. It allows a (reasonably) easy interface from almost any language, and allows you to choose whatever language you feel is best for writing the actual web service. However, in doing it this way, you're paying the cost of an extra network call per API call. If you put the web service on the same host (or local network) as the database, you can minimize the cost of the network call from the web service to the database, which mitigates the cost of the extra call to the API.
If the API has business logic in it, there's two via approaches...
You can write the API as a library that can be used from multiple languages. C is a good choice for this because most languages can link C libraries, but the languages you expect to use it from can have a large impact, too. For example, if you know it's always going to be used by a language hosted on the JVM, the any JVM language it probably a reasonably good choice.
Another choice is to use a hybrid of the two. A REST API for database access, plus a business layer library written in multiple languages. The idea being that you have business logic on the application end, but it's simple enough that you can write a "client library" in multiple languages that knows how to call out to the REST API and then apply business logic to the results it gets back. Assuming the business logic isn't too complex (ie, limited to ways to merge/view the database data), then this isn't a bad solution.
The benefit is that it should be relatively easy to supply one "default" library that can be used by many languages, plus other language specific versions of the library where you have time available to implement them. For cases where figuring out what calls need to be made to the database, and how to combine the results, can be complicated, I find this to be a reasonably good solution.
I would resort to webservices. Doesn't matter what language you use as long as you have a framework to interact with webservices you are good. Depending on your needs you could expose a simple REST API or go all the way with SOAP/WSDL and the likes.
Related
I would like to know in which scenarios would embedding scripting language in my C projects help me.
I have heard about lua which the developers embed in their projects to extending their software applications but why developers prefer to extend their applications using some scripting engine rather then the primary language?
This mostly comes down to who ends up using your applications and what they are using it for. If the application needs no per-user customization, then there's no need for scripting. New functionality can be added normally as part of the application.
However, like in game engines, if the users need to create/script custom behaviour into the application there has to be some way for them to do so. You could try to make your users write their scripts in the language of the application, however in the case of C and many other languages, this requires the code for the application to be recompiled (Not to mention the fact that your users might not be programmers and could benefit from a more high level scripting language).
By adding a scripting engine, you allow your users to add their own (limited) functionality to the application without needing to understand or recompile the entire codebase.
tl;dr Scripting engines make sense if your users need routinely to add custom behaviours to the application.
Regarding your "scenario" question : adding a script engine to your standalone (C or C++ for instance) application is the most direct way to combine the performance of a dedicated engine with the expertise of your power users.
No matter the field of you standalone application, thanks to the ability to write scripts, a power user will usually be able to get the most of it by creating dedicated or project-centric workflow.
The script interface brings a safe and secure environment that is suited for such users whose primary skills is generally not to deal with C/C++.
This leads to your second question : a script API perfectly (using Lua or Squirrel, for instance) makes sense if the primary language needs low-level programming skills. Typically, and application written in C++ will require your power users to write plugins using a C++ SDK.
On the opposite, if your standalone application is written in Python, the benefit of embedding Lua is far from obious in my opinion.
There are different systems (for different environments, languages) which helps to map schema to data structures in code, in databases, and do automatic glue code between them - ORM, etc.
Is there any common name/abbreviation (like ORM) for systems that in addition do mapping to user interface (in automatic/semi-automatic ways)?
Are there such systems? Of course it would require to select GUI templates for different parts of schema, maybe add a little logic at some places (declarative approach is preferable).
I remember that something similar was available at MS Access - it could automatically create UI Forms from schema or even queries. But I wonder if there are similar systems? (doesn't matter if it web GUI or desktop GUI).
I guess there can be library/framework for do this (Python, C#, Java, C++, PHP, etc), or stand-alone apps like MS Access.
You seem to be talking about scaffolding. Scaffolding is a way to display a very basic user interfaces (GUI) normally based on a database structure. Depending on the language, framework and implementation it may also provide basic Create, Read, Update, Delete (CRUD) actions on the GUI.
It is often frowned upon for anything other than a very rapid prototype. In fact, 99% of the time you will save more time and be better off learning a framework and doing things "properly".
Having said that, web frameworks seem to be the kings of scaffolding.
Ruby on Rails
Django
CakePHP
C#/ASP.NET
My advice would be to pick a well supported framework which favours convention over configuration and follow through some tutorials. You'll most likely end up in a better place.
I am an integration consultant and tend to use C and Lua in my spare time, unfortunately it is not my day job ;-(
Anyway, I tend to believe that a mixture of C and Lua is perfect for many "product" developments. I currently have an "adapter engine" built in pure C, but would like to actually move the adapter code to Lua....
For example, coding an EMAIL adapter in Lua is far easier than in C...yet I like the "engine speed of C"....
But now there is the big question of security risk in that the user can potentially add whatever he or she wants to the LUa scripts in production.....obviously there we could CHMOD the files...but is that really secure?
Ideally I want the C / Lua combination here....but now do I literally imbed the Lua code in the C application with a CHAR*....or do I issue a lua_dofile??
Thanks for the help
Lynton
First, one of the drawbacks to using C/Lua in production is it tends to be harder to find resources who can develop for these languages. C++ and JavaScript programmers are typically easier to find.
In terms of security, the key here is to use leading practices. Security is about risk reduction, there is no expectation one can achieve perfect security so you need to mitigate risk.
Here are my suggestions:
As with all middleware you need to use a hardened server. This is the first step, if the server is compromised using any platform you are in trouble. Middleware should NOT be in the DMZ.
You want to store the Lua code external to the compiled code (otherwise you lose the advantage of using Lua.) Make that storage as secure as you can. CHMOD is good, a secure DB server is better. The more secure the script store the more secure the system.
You can encrypt the Lua source - this is a trade off since it makes it a little harder to gain the advantage of easy updates and modification. You will probably need to implement decrypted script caching for performance.
Your security is as strong as your weakest link. If you provide a way to modify the Lua source via external access this will be an attack vector. Avoid this design if you can.
You should consider putting in change management checks. For example a separate place in the system where a checksum for each Lua file is stored. Then if an un-authorized change is made to a script you can abort functioning till the security breach is mitigated.
Other than the drawback I mentioned above, I don't think there is anything fundamentally flawed in your plan. If it can aid in making a good middleware system I would say go for it. Just mitigate the risk of your adapter scripts getting compromised as much as possible.
To expand on Donal's comment - given the popularity of node I would say that JavaScript is the the leading practice in scripted middleware right now. If you can handle learning a new scripting language I would say it would be a good idea given the support, popularity, and tools available.
Your primary requirement in terms of security is to ensure that the server cannot evaluate anything send by clients by any mechanism (not just direct evaluation, but also through supplying filenames). A lesser requirement is that they should also not have any mechanism to produce a message that allows other clients to evaluate unexpected things (i.e., avoid XSS trouble). If you can satisfy these requirements, you've got a safe server and the language(s) that it written in won't matter; using multiple languages is in fact a good idea as it lets you leverage the best of each.
It's also a good idea to use a carefully configured firewall, plenty of privilege restriction, some kind of DMZ proxy system to at least verify basic syntactic legality of messages, etc. These things are all just good practice. (Aim to configure things so the server can only just manage to do the service you want to provide.)
With sending email, there are a few other things to beware of. In particular, you do not want to be a conduit for spam, so you need to take care to ensure that arbitrary email headers cannot be constructed from client input and that the data formats you send out are non-executable (or that the data is constructed in a way that is non-evil). Rate limiting is also a good idea; unless your site is insanely popular, you shouldn't need to send more than a few messages a second across all clients. If you're ever sending only to a small set of addresses (e.g., to a fixed contact address) then you can relax these restrictions a bit (but still be careful of header injection). In all cases, route all email by a specialist email handling server instead of doing routing yourself as this avoids a whole lot of configuration difficulties.
My team is maintaining a huge Client Server win32 Delphi application. It is a client/server application (Thick client) that uses DevArt (SDAC) components to connect to SQL Server.
The business logic is often "trapped" in Component's event handlers, anyway with some degree of refactoring it is doable to move the business logic in common units (a big part of this work has already been done during refactoring... Maintaing legacy applications someone else wrote is very frustrating, but this is a very common job).
Now there is the request of a web interface, I have several options of course, in this question i want to focus on the VCL for the web (intraweb) option.
The idea is to use the common code (the same pas files) for both the client/server application and the web application. I heard of many people that moved legacy apps from delphi to intraweb, but here I am trying to keep the Thick client too.
The idea is to use common code, may be with some compiler directives to write specific code:
{$IFDEF CLIENTSERVER}
{here goes the thick client specific code}
{$ELSE}
{here goes the Intraweb specific code}
{$ENDIF}
Then another problem is the "migration plan", let's say I have 300 features and on the first release I will have only 50 of them available in the web application. How to keep track of it? I was thinking of (ab)using Delphi interfaces to handle this. For example for the User Authentication I could move all the related code in a procedure and declare an interface like:
type
IUserAuthentication= interface['{0D57624C-CDDE-458B-A36C-436AE465B477}']
procedure UserAuthentication;
end;
In this way as I implement the IUserAuthentication interface in both the applications (Thick Client and Intraweb) I know that That feature has been "ported" to the web. Anyway I don't know if this approach makes sense. I made a prototype to simulate the whole process. It works for a "Hello world" application, but I wonder if it makes sense on a large application or this Interface idea is only counter-productive and can backfire.
My question is: does this approach make sense? (the Interface idea is just an extra idea, it is not so important as the common code part described above) Is it a viable option?
I understand it depends a lot of the kind of application, anyway to be generic my one is in the CRM/Accounting domain, and the number of concurrent users on a single installation is typically less than 20 with peaks of 50.
EXTRA COMMENT (UPDATE): I ask this question because since I don't have a n-tier application I see Intraweb as the unique option for having a web application that has common code with the thick client. Developing webservices from the Delphi code makes no sense in my specific case, so the alternative I have is to write the web interface using ASP.NET (duplicating the business logic), but in this case I cannot take advantage of the common code in an easy way. Yes I could use dlls maybe, but my code is not suitable for that.
The most important thing that you have to remember is this:
Your thick client .EXE process is used by one person at a time (multiple persons will have multiple instances of that .EXE).
Your intraweb .EXE process will be used by many persons at a time. They all share the same instance of the process.
That means your business logic must not only be refactored out into common units, the instances of the business logic must be able to reside into memory multiple times, and not interfere.
This starts with the business logic that talks to the database: you must be able to have multiple database connections at the same time (in practice a pool of database connections work best).
In my experience, when you can refactor your business logic into datamodules, you have a good starting point to support both an Intraweb and a thick client version of your app.
You should not forget the user interface:
Thick clients support modal forms, and have a much richer UI
Web browsers support only message dialogs (and then: those are very limited), all fancy UI stuff costs a lot of development time (though for instance, TMS has some nice components for Intraweb)
Then, to top it off, you have to cope with the stateless nature of the HTTP protocol. To overcome this, you need sessions. Intraweb will take care of most of the session part.
But you need to ask yourself questions like these:
what should happen if a user is idle for XX minutes?
how much session state can I store in memory? and what if it doesn't fit?
what do I do with the session state that does not fit into memory?
This is just a start, so let use know when you need more info.
If it gets very specific to your app, you can always contact me directly: just google me.
--jeroen
I think if you move your application to n-tiers will be a better solution, it will be easier after that to be used by the desktop and web applications.
you already made the first part by decoupling the business logic from the presentation, you can use RemObject SDK or DataSnap that bundled with Delphi.
after that you will have working desktop application, and you can use Intrawebm Asp.net or what ever for web part, and in this way you will not have to duplicate the business logic again for the web part.
usually converting desktop application to web not easy as you thought, because they work in different environment, and you need to build each one as it's nature.
Our company has a point of sale system with many extras, such as ordering and receiving functionality, sales and order history etc. Our main issue is that the system was not designed properly from the ground up, so it takes too long to make fixes and handle requests from our customers. Also, the current technology we are using (Progress database, Progress 4GL for the language) incurs quite a bit of licensing expenses on our customers due to mutli-user license fees for database connections etc.
After a lot of discussion it is looking like we will probably start over from scratch (while maintaining the current product at least for the time being). We are looking for a couple of things:
Create the system with a nice GUI front end (it is currently CHUI and the application was not built in a way that allows us to redesign the front end... no layering or separation of business logic and gui...shudder).
Create the system with the ability to modularize different functionality so the product doesn't have to include all features. This would keep the cost down for our current customers that want basic functionality and a lower price tag. The bells and whistles would be available for those that would want them.
Use proper design patterns to make the product easy to add or change any part at any time (i.e. change the database or change the front end without needing to rewrite the application or most of it). This is a problem today because the Progress 4GL code is directly compiled against the database. Small changes in the database requires lots of code recompiling.
Our new system will be Linux based, with a possibility of a client application providing functionality from one or more windows boxes.
So what I'm looking for is any suggestions on which database and/or framework or programming language(s) someone might recommend for this sort of product. Anyone that has experience in this field might be able to point us in the right direction or even have some ideas of what to avoid. We have considered .NET and SQL Express (we don't need an enterprise level DB), but that would limit us to windows (as far as I know anyway). I have heard of Mono for writing .NET code in a Linux environment, but I don't know much about it yet. We've also considered a Java and MySql based implementation.
To summarize we are looking to do the following:
Keep licensing costs down on the technology we will use to develop the product (Oracle, yikes! MySQL, nice.)
Deliver a solution that is easily maintainable and supportable.
A solution that has a component capable of running on "old" hardware through a CHUI front end. (some of our customers have 40+ terminals which would be a ton of cash in order to convert over to a PC).
Suggestions would be appreciated.
Thanks
[UPDATE]
I should note that we are currently performing a total cost analysis. This question is intended to give us a couple of "educated" options to look into to include in or analysis. Anyone who could share experiences/suggestions about client/server setups would be appreciated (not just those who have experience with point of sale systems... that would just be a bonus).
[UPDATE]
For anyone who is interested, we ended up going with Microsoft Dynamics NAV, LS Retail (a plugin for the point of sale and various other things) and then did some (and are currently working on) customization work on top of that. This setup gave us the added benefit of having a fully integrated g/l system, which our current system lacked.
Java for language (or Scala if you want to be "bleeding edge", depending on how you plan to support it and what your developers are like it might be better, but also worse)
H2 for database
Swing for GUI
Reason: Free, portable and pretty standard.
Update: Missed the part where the system should be a client-server setup. My assumption was that the database and client should run on the same machine.
I suggest you first research your constraints a bit more - you made a passing reference to a client using a particular type of terminal - this may limit your options, unless the client agrees to upgrade.
You need to do a lot more legwork on this. It's great to get opinions from web forums, but we can't possibly know your environment as well as you do.
My broad strokes advice would be to aim for technology that is widely used. This way, expertise on the platform is cheaper than "niche" technologies, and it will be easier to get help if you hit a brick wall. Of course, following this advice may not be possible if you have non-negotiable technology already in place at customers.
My second suggestion would be to complete a full project plan, with detailed specs and proper cost estimates, before going with the "rewrite from scratch" option. Right now, you're saying that it would be cheaper to rewrite the system than maintain it, and you don't really know how much it would cost to re-write.
I suggest you use browser for the UI.
Organize your application as a web application.
There are tons of options for the back-end. You can use Java + MySQL. Java backend will save you from windows/linux debate as it will run on both platforms. You won't have any licensing cost for both Java and MySQL. (Edit: Definitely there are a lot of others languages that have run-times for both linux & windows including PHP, Ruby, Python etc)
If you go this route, you may also want to consider Google Web Toolkit (GWT) for creating the browser based front-end in a modular fashion.
One word of caution though. Browsers can be pesky when it comes to memory management. In our experience, this was the most significant challenge in doing browser based POS You may want to checkout Adobe Flex that runs in browser but might be more civil in its memory management.
What is CHUI? Character-UI, as in VT terminals? Or even 3270 style?
It sounds like you need a 3-tier system - the database backend, a middle-layer that runs the bulk of the back-end business processes, and a front-end layer for the CHUI / GUI / data-gateway.
All three layers can reside on one machine; or you can distribute the tiers out to various servers. The front-end layer would control the actual terminals, whether they are VT-terminals, or a web-browser, or a custom-written 'client' application.
Make sure you have considered the hardware needs here -- are you going to have barcode scanners, cash drawers, POS debit/credit terminals, et cetra? If you are using a standard browser, it might be hard to reliably integrate those items. (At the very least, you're likely going to have to write special applets to handle them.)
Finally, consider the possibility of a thin-client technology on Windows. It greatly simplifies system management, since you only have to upgrade the software centrally. Thin-client PC's are cheap -- sub $200.
Golden Code Development (see www.goldencode.com) has a technology that does automated conversion of Progress 4GL (the schema and code... the entire application) to a Java application with a relational database backend (e.g. PostgreSQL). They currently support a very complete CHUI environment and they do refactor the code. For example, the conversion separates the UI, the data model and the business logic into separate Java classes. The entire result is a drop-in replacement that is compatible with the original (users don't need retraining, processes don't need to be modified, the data is migrated too). This is possible because they provide an application server and a set of runtime classes that provide that compatibility. The result of the automated conversion is not something that needs further editing before you can compile and run it. True terminal support is included so hardware terminals still work (it requires a small JNI library to access NCURSES from Java). All the rest of the code in the runtime is pure Java. No Progress Software Corp technology is used in the resulting system and it runs on Linux.
At least one converted system is already in production, running a 24 by 7 mission critical environment. It is a converted ERP system that their mid-sized pilot customer uses to run their entire business.