Where can I find examples of code, written in "Unified Parallel C"?
I also interested in normative documents about this language (standards, reference manuals, online-accessible books and courses). What extensions were added to C to get UPC?
Is this dialect alive or dead?
UPC is still alive as a research project at UC Berkeley and is probably in use by high-performance computing facilities and national labs associated with the research group. You can get the latest version, released November 2009, here. Documentation, sample code, etc. can be found here. The language spec is here; there are many extensions on top of C99 syntax to support threads and inter-thread communication as first-class entities (perhaps not in the strictest sense of "first class", but certainly more so than in C, where you call out to opaque libraries to do synchronization and communication).
UPC is still in use some places; I am not an expert, but from what I can tell using UPC consists of a GCC-based compiler, a runtime, and a messaging layer GASNet that runs on top of your networking stack. These seem to be relatively well-supported on the types of machines for which you might want something like UPC. I wouldn't call it a huge developer community, but if you administer a large parallel machine, it's worth giving it a shot. If you just want something to work on your laptop, desktop, or server box, there is a wide variety of other parallel programming models with commercial support, tools, etc.
Other than the berkley UPC runtime based UPC implementations:
Berkley UPC-to-C translator http://upc.lbl.gov/download/source.shtml
and
gcc-upc http://www.gccupc.org)
there are also proprietary compilers by HP h30097.www3.hp.com/upc/
and Cray.
I believe that Cray and HP UPC versions both have progress guarentees, where as Berkley derived ones don't (This means that if you write spinlock style code, you have to explicitly call bupc_relax to prod teh runtime into precessing a remote write that would get you out of the loop)
UPC is still very much alive in the HPC community.
The new 1.3 version of the language/library specification was released in Nov 2013 and is available here:
https://upc-lang.org/upc-documentation
The major UPC compilers have already (or soon will) release 1.3 compliant versions.
The UPC specification is written as a "diff" against C99, so it's easy to see what's been added. At a high level it's linguistically a few new keywords and some extensions to the type system to accommodate the PGAS memory model and language-level SPMD threading. There is also a large (and growing) set of libraries for supporting common HPC needs in UPC applications.
There's lots more information on UPC at:
http://upc.lbl.gov/
where you can download a free, open source compiler that runs on vast variety of systems. The runtime source download includes a library of UPC example programs, benchmarks and test suites.
Related
I want to use CORBA for a C application and thus I wanted to compare the various implementations available on the internet, e.g. the ones mentioned in the title. However, I can't find any comparison that is remotely up-to-date, like this and this.
My core requirements are as follows:
Client and server can be created in different languages and compiled on different OS's.
The server must be able to push calls.
No data overhead, i.e. Web Service with their relatively huge xml messages are out of question.
The CORBA implementation must be available under a free license that allows creation of commercial applications.
Can anyone here give advice? I know CORBA is old, as I have read
Why has CORBA lost popularity? and Is CORBA legacy?, so I don't want to start another discussion on how horribly bloated CORBA is etc. I'd just like to know which implementation fits my requirements the best, but if you know of something that's better than CORBA, feel free to post it here, too.
Cheers
Alex
I think the old comparisons are very old and not relevant any more. Of the three ORBs you mention TAO is actively maintained and supported, it is the leading ORB supporting the IDL to C++ language mapping.
TAO and MICO are both C++ ORBs, Orbit has support for C, so if your requirement is only pure C, than Orbit is the only option. If you can wrap again a C++ ORB in C, than TAO would be the option.
If you are looking for a more modern implementation, than check the new IDL to C++11 language mapping, which is very easy to use. There is one implementation already on the market, called TAOX11, which uses parts of TAO for its core, but that is a commercial ORB with developer licenses and royalty free runtime licenses.
I am getting interested in the Oberon language and I would like to know: is the language actually used by common programmers or is it still only used by researchers? Is it production-ready? What I have in mind are non-scientific applications requiring GUI support and possibly Internet connectivity (at least client-side POP3 and SMTP functionality).
Also, which of the Oberon flavors would you recommend for my needs (Oberon2, Active Oberon, etc)? The simpler, the better, as long as it is well maintained and has some community.
If possible, I would like to run my applications in a conventional host environment (Windows or Linux), without the need for a special runtime environment or a special operating system.
Thanks
BlackBox has some of what you want, runs on flavors of Windows.
There are also some environments that compile to Java bytecode and target the JVM.
Look at POW, and Gardens Point Component Pascal.
I happen to be using some command-line only tools that are Oberon Compilers.
OO2C is an Oberon to C compiler (but the output is not for human consumption).
Ofront is an Oberon to Human-Readable C, but I haven't yet set up a linux box to run it on. (otherwise, it is supposed to run inside of BlackBox on Windows).
There is also Oxford Oberon Compiler by Professor Spivey. A VERY enjoyable Compiler that compiles to a Virtual Machine, but the whole object code is a self-contained application (albeit command -line).
It is a VERY small download, meant for an educational environment, keeps everything CLEAN, and works well for prototyping some of the grunt work or procedures/modules of your code. It also is supposed to allow bitmap drawing in XWindows in Black and White only, probably for drawing graphs, etc, but I have not had an opportunity to use that feature yet.
It has a GUI-based debugger, profiling, and some other interesting tools, and still is very small by comparison to most modern compilers like gcc. It is also totally stand alone.
Works on Mac, Win, Linux, and has source.
By comparison, OO2C took me about a day of futzing and compiling to get it going (but it is working).
I don't have a Windows box right now, so I can't run my copy of BlackBox, but it had a full GUI, and lots of Source code available at the Component Pascal Collection website.
http://www.zinnamturm.eu/index.htm
If you are looking for source code you should also check out that site in hopes you don't have to reinvent the wheel.
Really a joy to step into Oberon after having to fight C/C++ all day long to get simple stuff done.
OBNC is a new compiler for the latest version (2016) of the original Oberon language by Niklaus Wirth. It compiles via C and makes it easy to interface to existing C libraries.
https://miasap.se/obnc/
Given that Oberon [language] was developed as a complete [operating-]system, and that ETH's CS department ran ALL its computers (even the secretary's) on it I should think it is application-ready. This according to the following PDF:
http://www.ics.uci.edu/~franz/Site/pubs-pdf/BC03.pdf
is the language actually used by common programmers or is it still only used by researchers?
There was/is little use of the original Oberon language outside academia; there was some industrial adaptation of Oberon dialects like e.g. Component Pascal.
Is it production-ready?
Depends on your requirements. Given todays expectations of software developers the (original) language and available toolchains seem very minimalistic.
non-scientific applications requiring GUI support and possibly Internet connectivity ... in a conventional host environment... which of the Oberon flavors would you recommend for my needs?
GUI support and network programming in a conventional host environment is e.g. supported by https://blackboxframework.org as already mentioned, which uses a language related to Oberon.
You could also have a look at https://github.com/rochus-keller/Oberon which includes a platform independend IDE with semantic navigation and a source-level debugger, plus a platform independent foreign function interface as a language extension which allows you to directly use any C shared library, and thus reuse the plethora of existing proven GUI or network libraries out there without having to program in C. It also offers a modern, lean syntax variant without all the semicolons and capitalized keywords, which should appeal especially to younger developers; but of course also the traditional syntax is supported, even mixed modern/traditional syntax projects.
I'm trying to learn how to get information from PCI devices in Mac OS X. So far I haven't been able to find anything about it in sysctl(3) and it seems the old devinfo(3) facilities from BSD are not available.
I need a C facility for this, I know there is an I/O kit framework for Objective C but I cannot use this.
The specific information I'm after is pretty basic, just vendor and device ID's.
I/O Kit is the system framework for device information. It is not an Objective-C API; instead, Apple use a restricted subset of C++. Quoting the I/O Kit Fundamentals document,
Apple considered several programming languages for the I/O Kit and chose a restricted subset of C++.
C++ was chosen for several reasons. The C++ compiler is mature and the language provides support for system programming. In addition, there is already a large community of Macintosh (and BSD) developers with C++ experience.
The restricted subset disallows certain features of C++, including
Exceptions
Multiple inheritance
Templates
Runtime type information (RTTI)—the I/O Kit uses its own implementation of a runtime typing system
These features were dropped because they were deemed unsuitable for use within a multithreaded kernel. If you feel you need these features, you should reconsider your design. You should be able to write any driver you require using I/O Kit with these restrictions in place.
If you cannot use C++ then one alternative is to have your C program call /usr/bin/ioreg and parse its results.
Edit: you might want to take a look at the Accessing Hardware from Applications document. It looks like accessing the I/O registry can be done with C code for the most part (if not all), with a bit of Core Foundation.
Do any proposed, or implemented languages fit in the same (enormous) niche as C, with the intention of being an alternative, while maintaining all the applicability to OS, high performance, embedded and other roles?
There are quite a number of languages that were explicitly designed to fit all of that niche:
BitC
Cyclone
Forth
Mesa
CPL
BCPL (simplified version of CPL, implementation language of MULTICS)
B (Ken Thompsons first try at a systems programming language, based loosely on BCPL, precursor to C)
Ada
Go
D
C++
Modula-2 (specifically designed for the Lilith personal computer as a successor to Pascal for systems programming, also used by IBM as the implementation language for the original OS/400)
Oberon (specifically designed as a simpler successor to Modula-2)
Component Pascal (object-oriented successor to Oberon, despite the name it is not a direct successor to Pascal)
Modula-3 (despite the name not a successor to Modula-2 but an independent development)
Sing# (the implementation language of Microsoft Research's Singularity Research OS)
Limbo (language for the Inferno operating system (successor to Plan 9 (successor to Unix)))
Ooc
Erlang (maybe not for operating systems, but embedded realtime systems, especially in the telco industry (phone switches etc.), also lately (somewhat surprising to Erlang's inventors, actually) web servers, databsase systems, etc.)
Interestingly, there are also a number of languages that were not specifically designed to fill that niche, but that have been very successfully used in that niche:
Smalltalk (several Smalltalk OSs, embedded systems, microcontrollers, realtime systems, most famous: the Tektronix TDS500 series of digital oscilloscopes)
Lisp (several Lisp OSs, embedded systems, microcontrollers, some NASA spacecraft)
Java (several Java OSs (JNode, NewOS), embedded systems, microcontrollers, smartcards)
C# (several OSs (Cosmos, SharpOS), Mono is used in High-Performance Computing)
Haskell (the House OS, the seL4 verified microkernel)
Pascal (MacOS)
There's also a lot of languages that have not yet been used in that niche, but that certainly could be. (Mostly that is because those language communities themselves have been so indoctrinated by the "you can only write operating systems in C" bullshit that they actually believe their own language to be unusable.)
Ruby
Python
ECMAScript (which is actually used for writing high-performance webservers lately)
[Note that for each one of the three categories I listed there are literally thousands more languages that fit in there.]
In fact, one sometimes gets the feeling that languages which are not specifically designed for, say, operating system programming are actually better for that kind of thing. Compare, for example, the level of innovation, the stability, number of security holes, performance in something like a Smalltalk OS from the 1970s and Windows or OSX from 2010.
Personally, I believe that this is based on some deep-seated myths in the systems programming community. They believe that systems programming in a language with, say, strong typing, type safety, memory safety, pointer safety, automatic storage management is impossible and that the only way to get performance or realtime guarantees is to forego powerful abstraction facilities. However, it turns out that when you try to design a programming language for humans instead of machines, then humans can actually understand the programs they wrote, find security holes, fix bugs and locate and fix performance bottlenecks much better in a 1 line monad comprehension than in a 100 line for loop.
For example SqueakNOS, which is a variant of the Squeak Smalltalk system that runs without an OS (in other words: it is the OS) has pretty much all of the features that you would expect from a modern OS (graphical user interface, ...) plus some that you don't (embedded scripting language that can modify every single piece of the OS at runtime) and weighs in at just 300k SLOC and boots in less than 5 seconds while e.g. Windows weighs in at 50 million SLOC.
The obvious one is C++.
Does everything you describe, but extends C quite a bit with other features (Object Oriented, etc.).
Another interesting system programming language from Google: Go
BitC is a specific attempt. Here's a great article on alternatives to C, and why they've failed.
Ada is probably the most widely used language in this space apart from C.
It is designed above all to produce reliable bug free code, but, most Ada compilers produce well optimised effiecient machine code as well.
For a while this language was compulsary for Department of Defence projects and it is still widly used in avaionics, radar, navigation and weapons control systems.
You could consider D. From the homepage:
D is a systems programming language. Its focus is on combining the power and high performance of C and C++ with the programmer productivity of modern languages like Ruby and Python. Special attention is given to the needs of quality assurance, documentation, management, portability and reliability.
The D language is statically typed and compiles directly to machine code. It's multiparadigm, supporting many programming styles: imperative, object oriented, and metaprogramming. It's a member of the C syntax family, and its appearance is very similar to that of C++.
Ada and in most cases Objective-C.
freepascal.
pascal was created even before C, when there was not enough RAM and CPU for virtual machines and is still used for exactly the same porpuses than C.
modern incarnations of pascal namely Delphi and freepascal added OO and generics, following the evolution C++ has represented.
they share many concepts and design like pointers, direct allocation deallocation of memory, direct call of ASM inside programs, they are so similar that is not unusual to load dll or c/c++ code in pascal programs and vice versa.
probably looking at ancient languages like Basic it is possible to find implementations that are fitting the same niche of c.
languages follow platforms...
Do any proposed, or implemented languages fit in the same (enormous) niche as C, with the intention of being an alternative, while maintaining all the applicability to OS, high performance, embedded and other roles?
OSs were historically implemented in assembler. Later on development shifted to C, which initially was a sort of macro assembler.
Now most OSs are written mostly in C because it is pretty much only language which maintains some sort of assembler backward compatibility (e.g. one can map one to one lion share of the assembler found in the hardware specs into C). And libc is the primary interface - often is the only interface - between kernel and user-space. And yet the interface covers not everything: some things in kernel has to be accessed directly as no standard interface is (yet) provisioned. E.g. one has to use a C struct to pass parameters/retrieve results to/from ioctl.
That means that the use of C in application development is in greater part pushed by the simple fact that if you use C you get automatically access to all the features of kernel (OS) which is also written in C.
Only language which can somehow compete with C is the language which is based/compatible with C. The sole alternative known to me is the C++. In older times there were also relatively popular translators like p2c (Pascal to C): developer programs in one language, but the source code is automatically translated into C for compilation. But the translators were rather buggy and without knowledge of C often programs couldn't be debugged. So if you have to know some C anyway, why bother with the translators.
I personally (and many other developers I'm sure) using various languages often stumbled upon the problem that the OS has a feature, but the language used doesn't provide any facility to access it. That I think is the major deterrent factor for other language development. Even if you have bright a idea for a new language (which is likely to be incompatible with C, otherwise idea wouldn't be that bright) you end up with the burden to reimplement interface to the pretty much whole OS (and various must-have application libraries).
As long as (1) C remain the sole language for system programming and (2) OS interfaces are still evolving, all non-C-compatible languages on the other side of fence, application development, would be at greater disadvantage.
P.S. Actually that is one of the molds I hope LLVM/clang might break. clang is implemented as a reusable library theoretically allowing to mix languages. E.g. main source file can be in one language (and parsed by one front-end), but the #includes could be in C (and parsed by the clang).
Scala Native tries to be the functional-programming & functional-objects evolution of C's niche. Scala Native generates hardware machine code instead of JVM. https://scala-native.readthedocs.io/en/v0.4.0/
Once upon a time, a team of guys sat down and wrote an application in C, running on VMS on a VAX. It was a rather important undertaking and runs a reasonably important back-end operation at LargeCo. This whole shebang works so well that twenty-five years later it's still chugging along and doing it's thing.
Time passes and people retire and it so happens that the Last Man Standing has turned over the keys to a new generation who - we might imagine - are less than thrilled to find themselves caretakers of a system old enough to be their younger brother. Yet, as underwhelmed as they are by the idea of dealing with Ultra Legacy Systems, they can't justify the cost of replacing the venerable application.
LMS discovered that I habla unix and put this question to me. And since I habla unix but don't speak the C I shall summarize and put it to you. Long Story Short:
LMS wants to port LegacyApp, written in C. from VMS to unix. Resources? Any books he can read? People he can talk to?
The first question I'd need to ask is why, and I'd be leading the conversation in the direction of "Do you really need to port it off of VMS". There are a number of things worth mentioning about VMS:
-> VMS is still actively developed and maintained by HP. They just release V8.4 for Field Test last week (see http://h71000.www7.hp.com/openvmsft/).
-> VMS is available on new hardware; specifically HP's Integrity servers based on the Itanium processor.
-> VMS is also available on virtual platforms via the Charon Emulation products.
-> Popular estimates are that there are about 300,000 VMS systems still in active use today. LMS may be the last man at LargeCo, but he's far from the last man standing worldwide.
-> Lots of information out there, see openvms.org for example, to see lots of current information on VMS, all from current users.
OK - you still want to port off of VMS. How do you do it? Well, it depends on lots of stuff.
-> As others have said, how standard is the code? Chances are, not very. The more VMS-isms, the more difficult the job. 'nuff said.
-> What is the database? If it's Oracle, probably not too tough to move to Oracle on some other platform. If it's some sort of custom DB based on RMS index files, then you've got more work to do, you'll need to re-create that pseudo DB, or, understand it enough to replace it with some relational DB.
-> Besides C, what else is used to create the application? What's on the front end? DECforms? FMS? Is there a transaction engine, e.g. ACMS? RTR? These things will have a huge impact on the feasibility and effort required to port to UNIX.
-> What other products are involved? Are there any 3rd party libraries being used? Are there 3rd party products in use that are critical to the application or functionality?
-> Is this system clustered? If so why? You'll need to meet those same goals with the UNIX box.
-> There are companies out there that will help you do it, and claim to have tools to make it easier, but my experience is that these companies tend to be selling you more services than products (i.e. you need to hire them to use the tools. It'll be expensive).
The book UNIX for OpenVMS Users will give the VMS novice some help in understanding VMS, but, as the title says, the book is really intended for the opposite purpose.
Everything written on VMS uses lots of VMS specific stuff it was just so convenient.
There are a few companies that sell compatibility libs to make the port easier - they wont be cheap though, VMS tended to be used where reliability mattered more than cost.
The other option is to run openVMS on some modern hardware, possibly in a VM.
I am sure Brian has made his decision by now, but for my sins of working for many years in DEC OpenVMS language support (yes, some people had this dubious honour) the real question I would have asked a customer such as Brian is: is it a real-time application or not? If it is the former, then it would be heavily dependent on many VMS system services which would rule out a 'port' and indicate a re-write. If it were the latter then the frequency of VMS system services should (possibly) be limited and make a port viable.
The acid test for me, would be to SEARCH *.c "SYS$", "LIB$" i.e. to search all of the C source files for "SYS$" and "LIB$" tags which prefix VMS system services. If the count for these are in the 10s then a port is probably likely, between 10 and 100 makes it possibly likely, but over a 100 makes a successful port highly unlikely.
Hope this helps
You have several choices.
Get the OpenVMS source, and continue to maintain Open VMS as if it were a Linux distribution. Some folks don't mind keeping up with Linux distributions and OpenVMS distributions. It can be done.
Try to recompile the VMS C into Linux. This can be trivial if the C used only standard libraries. This can be very, very difficult if the C used a lot of VMS libraries.
Once you have facts at your fingertips, you can reevaluate this course of action. Since you didn't list a bunch of VMS library methods this program uses, it's impossible to tell how entangled it is with the OS.
This may be trivial or impossible. It's difficult to tell without analysis of the source.
Write bridge libraries from VMS to Linux. If your program only does a few VMS things, this isn't very difficult. If your program does extensive VMS things, this is craziness.
The bridge -- in the long run -- is a terrible idea. Managers love it, however.
An alternative is to replace the VMS library calls with proper, portable Linux calls rather than write bridges. This is better in the long run, because it excises the non-portable features of the program.
Rewrite it from scratch in Python. That is usually simpler than trying to port the C code. It will be shorter, cleaner, simpler, and portable.
If you're willing to keep running VMS in a VM, you can look into CHARON-VAX ( http://www.charon-vax.com/ ). As previously mentioned, the ease of porting really depends a lot on how much of the VMS extensions were used; searching the source code for $ characters embedded in strings (usually with a 3-character leading substring, such as lib$gettime or dsc$descriptor or sys$foobar etc) will give you at least a basic idea of what VMS system functions are called and how likely they are to be portable, if the name is reasonably obvious.
If it ain't broke, don't fix it! Why port it or migrate the app if you don't have to? Why not run it on a current install of OpenVMS running on an HP Itanium server; that is assuming you wish to upgrade the hardware, which may not even be necessary if your VAX hardware is still running strong.
To learn C, you might as well drag it from the horse's mouth: "The C Programming Language" by its inventors, Kernighan and Ritchie.
I can recommend "The UNIX programming environment" by (again) Brian Kernighan; a more authoritative source you'll hardly find, and it teaches you both Unix/C idioms and a bit of C programming at the same time.
For more depth and detail on C, I heartily enjoyed a book by Peter van der Linden: "Expert C Programming - Deep C Secrets".
You'll also want to wrestle LMS for a library documentation of VMS-specific C functions with (of course) special emphasis on those actually used in the app. That's where your porting effort will be.
The job could be easy or difficult, depending on how much machine-specific cleverness and bit-twiddling is done, and how many VMS-specific system calls are used. It would be very good if word size was equal (in other words, if your VMS box has a word size of 32 bits, don't run the code on a 64 bit version of Unix!)
Brian, I'm not sure if LMS specified/cared to port C-code or the WHOLE process. As too often people think of languages out of scope of systems.
If there're was a process built on VMS, most likely it used at least scheduling/batch facilities, which are often scripted in DCL (rather simple and clear language, unlike shell or perl scripting).
So the cost of porting the whole process may be higher than originally perceived by your LMS. Add here the reliability aspect, given your crunches with C, which is nothing impossible, of course, with enthusiasm and determination.
If you want simply give the C-code a try, as previously posted, search it for the "$" hits. Or just cc it with all headers present, the basics of compile-link command should be enough.
Alternatively, this looks like a consultant's call, as indeed such jobs were abundant at the "exodus" time. All said VMS remains quite a robust platform (24x7 is a norm!), unless the harware dies, then there're still tons of "exodus" spares. GOOD LUCK!
About a year and a half later, maybe you've already figured out what to do. My organization has recently decided to stick with OpenVMS instead of switching to Linux even though the old guard recently left. We just couldn't argue with what we felt was a very stable and reliable system. We are currently switching from Alpha servers to Integrity servers for end of life reasons. HP has been very helpful with our transition.
For that matter, there may be Linux vendors out there who can help with the transition. Ask your new hardware vendor if they have any recommendations.
Depending on what languages you already know, C is not that hard to learn. I taught myself C in the course of learning C++ after finally prying myself loose from Pascal.
(VAX Pascal, plus Rdb/VMS, plus DCL formed a combination that was hard to beat.)
If the software is typical C, you'll spend more time learning the library functions than learning the language.
It's pretty lightweight stuff, but I went through the online tutorials for C++ that Microsoft makes available in conjunction with the express edition of Visual Studio for C++.
Here's the beginner's tutorial:
http://msdn.microsoft.com/en-us/beginner/cc305129.aspx
It's probably worth making the effort to ask why LMS wants to port the application to Unix. The answer may seem obvious, but properly exploring the reasons has its benefits. I would assume:
OpenVMS is an "ultra legacy platform", and for that reason alone is something that is not worth running an application on anymore;
It's tough to find anyone who is willing to maintain an application that runs on OpenVMS these days;
The hardware on-which OpenVMS runs is threatening to become moribund.
We have a similar challenge, but in our case the application in question not only runs on OpenVMS but is also written in COBOL. I would have to say that your situation is rosy in comparison given that your application is written in a cross-platform language.
In any case, I think if you're about to make a big decision like moving from OpenVMS to Unix it would be prudent to do a little due diligence. In your case, try to assess just how portable the code is--only then will you know what the scale of the effort is (worst case could quite easily be a multiple of best case). In C, code portability is mostly a function of the dependencies--are they "standard" or are they VMS-specific?
Our enquiries revealed that HP would be supporting OpenVMS on Itanium until at least 2022. There isn't necessarily a need to rush to another platform--perhaps you could keep things on OpenVMS whilst embarking on an effort to prepare the application for porting (make it less dependent on OpenVMS specifics).
VMS has a surprisingly healthy community and if it's the lack of Unix that's the issue, then maybe GNV could help bridge the gap?
Well u have a few options. if this code needs to be ported rather quickly, i would write a bridge library to emulate the vms libs. whener you get it back up and running on a *nix, then go through replacing the vms library calls with native/portable calls for *nix.
Also if there is a lot of optimizations in the code ie inline assembly and bit twiddling. then you will have to rewrite thi code, which will take an understanding of the VAX arch. also. be sure to check word size differences and endian differences