Running Java batch deployed in Liberty profile on z/OS - batch-file

Is the approach of running java batch programs in Liberty profile (supporting JSR352 specification) on z/OS relatively new to the market or been for a long time ?
The reason behind this question is because, am hearing that this is a
relatively new attempt by IBM and there are not much of live systems
in the market running in this approach. Is that true ?
Note: I understand that the JSR352 has been there for quite sometime but, my question is specific to its support by the Liberty profile in z/OS (mainframe)

Support for JSR-352 showed up in WebSphere Liberty as part of the Java EE7 support delivered in 8.5.5.6 which was back in June of 2015. It is supported on z/OS and all the other platforms supported by Liberty. There are some extra features supported only on z/OS (i.e. SMF recording, a z/OS-specific Command Line Interface).
Support in WebSphere traditional for IBM's proprietary Java Batch product (WebSphere Compute Grid) goes back at least a decade (on z/OS and distributed platforms).
There's a lot of information about Liberty JSR-352 support (especially on z/OS) starting from here: http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102544

#yathirigan Java would not be viable on a mainframe if it were not for zIIP processors. Compared to traditional mainframe batch languages like COBOL or PL/I it uses enormous amounts of resources, both CPU and memory. One thing for sure is that you need to make sure you've got got enough zIIP engines for your workloads because if Java workloads spill over onto GCPs you may find your monthly license charge has gone through the roof.

Related

What does "binary identical" mean?

In the context of SQL Server for embedded systems.
I read the following blog:
http://blogs.msdn.com/b/windows-embedded/archive/2014/04/01/microsoft-sql-server-2014-for-embedded-systems-build-your-foundation-for-better-data-analytics-and-operational-intelligence-in-next-generation-intelligent-systems.aspx
In my opinion, there have to be some differences - because otherwise there wouldn't be a version especially for embedded systems...?
Thank you ;)
It's an April Fools joke. To me the joke is the hollow and enterprisey language. They are not really saying anything in the entire post.
The post says the truth: SQL Server Enterprise is available for embedded devices. Just use the normal binaries and pay 7000$ per embedded CPU core :)

Unified Parallel C - examples and list of extensions

Where can I find examples of code, written in "Unified Parallel C"?
I also interested in normative documents about this language (standards, reference manuals, online-accessible books and courses). What extensions were added to C to get UPC?
Is this dialect alive or dead?
UPC is still alive as a research project at UC Berkeley and is probably in use by high-performance computing facilities and national labs associated with the research group. You can get the latest version, released November 2009, here. Documentation, sample code, etc. can be found here. The language spec is here; there are many extensions on top of C99 syntax to support threads and inter-thread communication as first-class entities (perhaps not in the strictest sense of "first class", but certainly more so than in C, where you call out to opaque libraries to do synchronization and communication).
UPC is still in use some places; I am not an expert, but from what I can tell using UPC consists of a GCC-based compiler, a runtime, and a messaging layer GASNet that runs on top of your networking stack. These seem to be relatively well-supported on the types of machines for which you might want something like UPC. I wouldn't call it a huge developer community, but if you administer a large parallel machine, it's worth giving it a shot. If you just want something to work on your laptop, desktop, or server box, there is a wide variety of other parallel programming models with commercial support, tools, etc.
Other than the berkley UPC runtime based UPC implementations:
Berkley UPC-to-C translator http://upc.lbl.gov/download/source.shtml
and
gcc-upc http://www.gccupc.org)
there are also proprietary compilers by HP h30097.www3.hp.com/upc/
and Cray.
I believe that Cray and HP UPC versions both have progress guarentees, where as Berkley derived ones don't (This means that if you write spinlock style code, you have to explicitly call bupc_relax to prod teh runtime into precessing a remote write that would get you out of the loop)
UPC is still very much alive in the HPC community.
The new 1.3 version of the language/library specification was released in Nov 2013 and is available here:
https://upc-lang.org/upc-documentation
The major UPC compilers have already (or soon will) release 1.3 compliant versions.
The UPC specification is written as a "diff" against C99, so it's easy to see what's been added. At a high level it's linguistically a few new keywords and some extensions to the type system to accommodate the PGAS memory model and language-level SPMD threading. There is also a large (and growing) set of libraries for supporting common HPC needs in UPC applications.
There's lots more information on UPC at:
http://upc.lbl.gov/
where you can download a free, open source compiler that runs on vast variety of systems. The runtime source download includes a library of UPC example programs, benchmarks and test suites.

Cross Platform System Usage Metrics

Is there a cross platform C API that can be used to get system usage metrics?
I've worked with libstatgrab before. Gets you some pretty useful system statistics for the main Unix-like variants and Windows through Cygwin (supposedly - never tried). Different OS's work so differently - especially when it comes to usage metrics - it may be challenging to get what you want. Even something as simple sounding as "free memory" can be tricky to act on in a cross-platform way. Perhaps if you narrow things down a bit, maybe we can find something.
Unfortunately not.
The C standard is pretty much limited to dynamic allocation, string manipulation, math and text I/O. Once you get beyond that, you need OS APIs which by definition are OS specific and not cross platform.
Depending on the metrics you want to collect, you may want to consider looking at PCP (Performance Co-Pilot). This is an open-source performance framework, originally developed at Silicon Graphics, which collects and collates a vast number of possible metrics from a vast number of sources, and lets you monitor them from anywhere.
Basically PCP would involve adding another 'layer' into your system -- for example, you might monitor a distributed cluster of mixed-OS machines, each with PCP installed locally; a set of 'agents' collect the performance data on each machine, and your code could then use libpcp to collect those metrics as required.
It's hard to say without knowing your exact usage scenario (if you're talking about something running seamlessly on end-users' machines, PCP may not suit but if you want to monitor machines which you control, and are happy to run the PCP service on them, it's an awesome solution).
We use PCP very happily to collect metrics from Windows and Linux boxes, as well as internal metrics from our application, and log them all centrally, report on them, monitor trends, etc.
The good old SNMP provides C libraries for the client and server side. If you prefer something newer you should try Prometheus. They have Node Exporter ready to use and the client libraries for many languages including C.
Also note that PCP (mentioned in Cowan's answer) supports the OpenMetrics standard since version 4, so if you decide to use Prometheus as the metric data collector extending PCP or writing a custom Prometheus client are better solutions that SNMP which also supported by Prometheus through SNMP Exporter, but more difficult to set up due to the way it handles MIBs and authentication.

Moving from VMS to Unix

Once upon a time, a team of guys sat down and wrote an application in C, running on VMS on a VAX. It was a rather important undertaking and runs a reasonably important back-end operation at LargeCo. This whole shebang works so well that twenty-five years later it's still chugging along and doing it's thing.
Time passes and people retire and it so happens that the Last Man Standing has turned over the keys to a new generation who - we might imagine - are less than thrilled to find themselves caretakers of a system old enough to be their younger brother. Yet, as underwhelmed as they are by the idea of dealing with Ultra Legacy Systems, they can't justify the cost of replacing the venerable application.
LMS discovered that I habla unix and put this question to me. And since I habla unix but don't speak the C I shall summarize and put it to you. Long Story Short:
LMS wants to port LegacyApp, written in C. from VMS to unix. Resources? Any books he can read? People he can talk to?
The first question I'd need to ask is why, and I'd be leading the conversation in the direction of "Do you really need to port it off of VMS". There are a number of things worth mentioning about VMS:
-> VMS is still actively developed and maintained by HP. They just release V8.4 for Field Test last week (see http://h71000.www7.hp.com/openvmsft/).
-> VMS is available on new hardware; specifically HP's Integrity servers based on the Itanium processor.
-> VMS is also available on virtual platforms via the Charon Emulation products.
-> Popular estimates are that there are about 300,000 VMS systems still in active use today. LMS may be the last man at LargeCo, but he's far from the last man standing worldwide.
-> Lots of information out there, see openvms.org for example, to see lots of current information on VMS, all from current users.
OK - you still want to port off of VMS. How do you do it? Well, it depends on lots of stuff.
-> As others have said, how standard is the code? Chances are, not very. The more VMS-isms, the more difficult the job. 'nuff said.
-> What is the database? If it's Oracle, probably not too tough to move to Oracle on some other platform. If it's some sort of custom DB based on RMS index files, then you've got more work to do, you'll need to re-create that pseudo DB, or, understand it enough to replace it with some relational DB.
-> Besides C, what else is used to create the application? What's on the front end? DECforms? FMS? Is there a transaction engine, e.g. ACMS? RTR? These things will have a huge impact on the feasibility and effort required to port to UNIX.
-> What other products are involved? Are there any 3rd party libraries being used? Are there 3rd party products in use that are critical to the application or functionality?
-> Is this system clustered? If so why? You'll need to meet those same goals with the UNIX box.
-> There are companies out there that will help you do it, and claim to have tools to make it easier, but my experience is that these companies tend to be selling you more services than products (i.e. you need to hire them to use the tools. It'll be expensive).
The book UNIX for OpenVMS Users will give the VMS novice some help in understanding VMS, but, as the title says, the book is really intended for the opposite purpose.
Everything written on VMS uses lots of VMS specific stuff it was just so convenient.
There are a few companies that sell compatibility libs to make the port easier - they wont be cheap though, VMS tended to be used where reliability mattered more than cost.
The other option is to run openVMS on some modern hardware, possibly in a VM.
I am sure Brian has made his decision by now, but for my sins of working for many years in DEC OpenVMS language support (yes, some people had this dubious honour) the real question I would have asked a customer such as Brian is: is it a real-time application or not? If it is the former, then it would be heavily dependent on many VMS system services which would rule out a 'port' and indicate a re-write. If it were the latter then the frequency of VMS system services should (possibly) be limited and make a port viable.
The acid test for me, would be to SEARCH *.c "SYS$", "LIB$" i.e. to search all of the C source files for "SYS$" and "LIB$" tags which prefix VMS system services. If the count for these are in the 10s then a port is probably likely, between 10 and 100 makes it possibly likely, but over a 100 makes a successful port highly unlikely.
Hope this helps
You have several choices.
Get the OpenVMS source, and continue to maintain Open VMS as if it were a Linux distribution. Some folks don't mind keeping up with Linux distributions and OpenVMS distributions. It can be done.
Try to recompile the VMS C into Linux. This can be trivial if the C used only standard libraries. This can be very, very difficult if the C used a lot of VMS libraries.
Once you have facts at your fingertips, you can reevaluate this course of action. Since you didn't list a bunch of VMS library methods this program uses, it's impossible to tell how entangled it is with the OS.
This may be trivial or impossible. It's difficult to tell without analysis of the source.
Write bridge libraries from VMS to Linux. If your program only does a few VMS things, this isn't very difficult. If your program does extensive VMS things, this is craziness.
The bridge -- in the long run -- is a terrible idea. Managers love it, however.
An alternative is to replace the VMS library calls with proper, portable Linux calls rather than write bridges. This is better in the long run, because it excises the non-portable features of the program.
Rewrite it from scratch in Python. That is usually simpler than trying to port the C code. It will be shorter, cleaner, simpler, and portable.
If you're willing to keep running VMS in a VM, you can look into CHARON-VAX ( http://www.charon-vax.com/ ). As previously mentioned, the ease of porting really depends a lot on how much of the VMS extensions were used; searching the source code for $ characters embedded in strings (usually with a 3-character leading substring, such as lib$gettime or dsc$descriptor or sys$foobar etc) will give you at least a basic idea of what VMS system functions are called and how likely they are to be portable, if the name is reasonably obvious.
If it ain't broke, don't fix it! Why port it or migrate the app if you don't have to? Why not run it on a current install of OpenVMS running on an HP Itanium server; that is assuming you wish to upgrade the hardware, which may not even be necessary if your VAX hardware is still running strong.
To learn C, you might as well drag it from the horse's mouth: "The C Programming Language" by its inventors, Kernighan and Ritchie.
I can recommend "The UNIX programming environment" by (again) Brian Kernighan; a more authoritative source you'll hardly find, and it teaches you both Unix/C idioms and a bit of C programming at the same time.
For more depth and detail on C, I heartily enjoyed a book by Peter van der Linden: "Expert C Programming - Deep C Secrets".
You'll also want to wrestle LMS for a library documentation of VMS-specific C functions with (of course) special emphasis on those actually used in the app. That's where your porting effort will be.
The job could be easy or difficult, depending on how much machine-specific cleverness and bit-twiddling is done, and how many VMS-specific system calls are used. It would be very good if word size was equal (in other words, if your VMS box has a word size of 32 bits, don't run the code on a 64 bit version of Unix!)
Brian, I'm not sure if LMS specified/cared to port C-code or the WHOLE process. As too often people think of languages out of scope of systems.
If there're was a process built on VMS, most likely it used at least scheduling/batch facilities, which are often scripted in DCL (rather simple and clear language, unlike shell or perl scripting).
So the cost of porting the whole process may be higher than originally perceived by your LMS. Add here the reliability aspect, given your crunches with C, which is nothing impossible, of course, with enthusiasm and determination.
If you want simply give the C-code a try, as previously posted, search it for the "$" hits. Or just cc it with all headers present, the basics of compile-link command should be enough.
Alternatively, this looks like a consultant's call, as indeed such jobs were abundant at the "exodus" time. All said VMS remains quite a robust platform (24x7 is a norm!), unless the harware dies, then there're still tons of "exodus" spares. GOOD LUCK!
About a year and a half later, maybe you've already figured out what to do. My organization has recently decided to stick with OpenVMS instead of switching to Linux even though the old guard recently left. We just couldn't argue with what we felt was a very stable and reliable system. We are currently switching from Alpha servers to Integrity servers for end of life reasons. HP has been very helpful with our transition.
For that matter, there may be Linux vendors out there who can help with the transition. Ask your new hardware vendor if they have any recommendations.
Depending on what languages you already know, C is not that hard to learn. I taught myself C in the course of learning C++ after finally prying myself loose from Pascal.
(VAX Pascal, plus Rdb/VMS, plus DCL formed a combination that was hard to beat.)
If the software is typical C, you'll spend more time learning the library functions than learning the language.
It's pretty lightweight stuff, but I went through the online tutorials for C++ that Microsoft makes available in conjunction with the express edition of Visual Studio for C++.
Here's the beginner's tutorial:
http://msdn.microsoft.com/en-us/beginner/cc305129.aspx
It's probably worth making the effort to ask why LMS wants to port the application to Unix. The answer may seem obvious, but properly exploring the reasons has its benefits. I would assume:
OpenVMS is an "ultra legacy platform", and for that reason alone is something that is not worth running an application on anymore;
It's tough to find anyone who is willing to maintain an application that runs on OpenVMS these days;
The hardware on-which OpenVMS runs is threatening to become moribund.
We have a similar challenge, but in our case the application in question not only runs on OpenVMS but is also written in COBOL. I would have to say that your situation is rosy in comparison given that your application is written in a cross-platform language.
In any case, I think if you're about to make a big decision like moving from OpenVMS to Unix it would be prudent to do a little due diligence. In your case, try to assess just how portable the code is--only then will you know what the scale of the effort is (worst case could quite easily be a multiple of best case). In C, code portability is mostly a function of the dependencies--are they "standard" or are they VMS-specific?
Our enquiries revealed that HP would be supporting OpenVMS on Itanium until at least 2022. There isn't necessarily a need to rush to another platform--perhaps you could keep things on OpenVMS whilst embarking on an effort to prepare the application for porting (make it less dependent on OpenVMS specifics).
VMS has a surprisingly healthy community and if it's the lack of Unix that's the issue, then maybe GNV could help bridge the gap?
Well u have a few options. if this code needs to be ported rather quickly, i would write a bridge library to emulate the vms libs. whener you get it back up and running on a *nix, then go through replacing the vms library calls with native/portable calls for *nix.
Also if there is a lot of optimizations in the code ie inline assembly and bit twiddling. then you will have to rewrite thi code, which will take an understanding of the VAX arch. also. be sure to check word size differences and endian differences

About mobile game porting

I would like to ask a few questions regarding mobile game porting...
Let say if I have a simple 2D C++ game engine and have a PC game based on that engine and I want to port it to different mobile platforms BREW, J2ME, iPhone, Android, Symbian, etc..
Do I need to re-code the engine and the game for each platform? or is there an easier and more efficient way? I am sure the process is complicated since different phones have different graphic/processor/memory/etc. I am just curious about the overview of mobile game porting process. :)
Thanks!
There are several ways of attacking mobile game porting. First of all, until very recently it was mostly BREW and J2ME. The iPhone, Android and BlackBerry are changing this landscape and making the impossible task of mobile game porting even more impossible. I worked in 3rd party mobile game development for many years until recently. I watched BREW vanish and saw publishers completely focus on J2ME as the cost of porting is strangling the industry. There are estimates to its cost, both time and money, and it seems to bell curve around 50-60% of the total development cost for each game is just porting.
At our company, we handled porting by having two engines that paralleled each other, one in BREW, one in J2ME. We never supported Symbian as Symbian development does not make any money. It is mainly for high-end tech demos that might be on one or two devices, nothing that could reach the mass market. Plus, most Symbian phones supported J2ME.
We would be required by publishers to provide any where from 7-23 reference builds of the game, targeting many different devices, in both BREW and J2ME. Just before moving on, publishers were also starting to require a J2ME touch screen reference version, and an iPhone SKU was being left as "to be determined" based on the final product and how cost effective an iPhone version would be at that time. The reference versions would then be passed on to a porting house to translate the different references to the thousands of other required SKUs.
Companies like Gameloft still brute force their way through porting. That's why Gameloft's games are constantly at a higher quality than the rest of the industry. However, it is just not possible for smaller companies to attack the problem this way due to costs. Not everyone can afford an office in Beijing with 5000 developers.
There are many companies out there developing engines to cut porting costs. Mobile-Distillery is one I was in contact with quite a lot, but we ended up never using it. So, I can't vouch for them. The problem here is that you will be at the mercy of another companies engine. Performance could be problematic due to the fact that it is being built to target thousands of SKUs. Plus, you really have little control over the low level implementation of your game in this instance. The end result seems to be a game that targets the lowest common denominator of phones.
Finally, a lot of developers are just abandoning the idea of supporting all mobile platforms. There is a huge flood on games on the iPhone because 1) it requires only targeting one platform and 2) there is a 70 percent profit share through the AppStore for developers. Through carrier releases, the percentage is not even comparable.
This of course depends on in which language your "simple engine" is written. Java is supposed to be easier to move between platforms, since it in effect is a platform of its own. If your engine is written in something more low-level, like C++, it will likely depend on platform-specific libraries for graphics and input, since C++ doesn't provide you with that.
I can personally recommend edgelib as a cross platform layer for mobile games and application development.
It is widely used, offers excellent performance and supports most of the important smartphone platforms in existence today: Symbian, Win Mobile, IPhone and others.
Most important: It's cheap and offers a free complete evaluation until you decide to produce commercial content with it.
You can find it at: http://www.edgelib.com
You can see a showcase of games and other projects based on this platform at the site.
Unfortunately, BREW is not supported though. And since it isn't a real open platform, it poses quite a challenge compared to other mobile OS platforms as it's development community is mostly professional and not very "talkative" in online terms...
This will perhaps change as the competition is getting more open by the minute.
I am not affiliated with Edgelib... this is just my personal opinion. :)
There is no free lunch. The platforms you are considering are not compatible, most of then allow you to run C/C++ code so in theory you could port the engine to some Standard, such as ANSI, or C99 and it would compile in most of the platforms PC, BREW however this does not take in account the libraries your engine might need. For example if your engine uses OpenGL then it would work in the PC, and some consoles but on Symbiam devices you need OpenGL-ES which is not exactly the same, so you need an abstraction for all libraries you use.
About J2ME and Android they are Java platforms, so no C/C++ can be run there without any special VM lib at least. In this case you need to port the C/C++ code to Java which can be overkiling.
So my answer to this is while you can make an abstraction to your libs and code using standards you might be able to use the same engine in several platforms as long you can use the same compiler for them.
Take a look for example to this engine CubicVR it allows you to compile the same engine for PC/Linux/MAC/iPhone (maybe)/Sony PlayStation Portable
What I have used for multiplatform development is to implement a hardware abstraction layer. The engine is coded in C++ but using a plain C interface to implement the system calls. This allows you to use full-fledged C++ for your game and engine and link with the system abstraction written in whatever language your platform needs. Symbian doesn't support 100% of the features of C++, and still has a few bugs, and IPhone API uses Objective C. C is compatible which most of the platforms you mention (well, not Java) and linking C is easier than C++, as there are less problems (ABIs and all that stuff).
Implementing a additional interface in C is a little slower, but will help you a lot when porting it to other platforms. Also, it allows you to have a Win32/Linux/Mac build besides the Symbian, BREW, etc. one. I have worked, mainly, with Symbian and N-Gage, and the debugging capabilities of those platforms are signifficantly behind Visual Studio or GDB. Iphone, on the other side, has a lot of cool tools to debug and profile your app.
If the engine you have is not written in JAVA or FLASH I don't see a way that you can use your engine out of the box without recoding it in a language which might run on a mobile phone.
If it is coded in JAVA or FLASH, I think you're main problem (beside the performance difference between a phone and a pc) is the main memory. As I've heared, the main memory is the biggest problem in mobile phone game development.
As others have pointed out, if your engine is built on top of some 'platform independent' framework such as Java, then it's supposedly easier than if it's written in a lower level language. However, neither Java nor Flash is, for example, supported on the iPhone/iPod touch, and it'll probably take a while until they are. On the other hand, if I recall correctly, the only SDK available for those platforms is Objective-C, which, guessing wildly, you didn't use to implement your engine.
In general, it depends on the application/game engine you've written. Most likely you'll have to change something, as very rarely are all features available on all platforms. J2ME graphics is a classic scenario of 'platform independent'-dependence, or so I've heard. How much you'll have to change solely depends on how portable your code is, i.e. how well did you separate out the parts that potentially need changing.
Unfortunately, this is the best answer I can give you.

Resources