I am new to parallel computing world. Can you tell me is it possible to run a c++ code uses MPI routines in my laptop with dual core or is there any simulator/emulator for doing that?
Most MPI implementations use shared memory for communication between ranks that are located on the same host. Nothing special is required in terms of setting up the laptop.
Using a dual core laptop, you can run two ranks and the OS scheduler will tend to place them on separate cores. The WinXP scheduler tends to enforce some degree of "cpu binding" because by default jobs tend to be scheduled on the core where they last ran. However, most MPI implementations also allow for an explicit "cpu binding" that will force a rank to be scheduled on one specific core. The syntax for this is non-standard and must be gotten from the specific implementations documentation.
You should try to use "the same" version and implementation of MPI on your laptop that the university computers are running. That will help to ensure that the MPI runtime flags are the same.
Most MPI implementations ship with some kind of "compiler wrapper" or at least a set of instructions for building an application that will include the MPI library. Either use those wrappers, or follow those instructions.
If you are interested in a simulator of MPI applications, you should probably check SMPI.
This open-source simulator (in which I'm involved) can run many MPI C/C++/Fortran applications unmodified, and forecast rather accurately the runtime of the application, provided that you have an accurate description of your hardware platform. Both online and offline studies are possible.
There is many other advantages in using a simulator to study MPI applications:
Reproducibility: several runs lead to the exact same behavior unless you specify so. You won't have any heisenbugs where adding some more tracing changes the application behavior;
What-if Analysis: Ability to test on platform that you don't have access to, or that is not built yet;
Clairevoyance: you can observe every parts of the system, even at the network core.
For more information, see this presentation or this article.
The SMPI framework can even formally study the correction of MPI applications through exhaustive testing, as shown in that presentation.
MPI messages are transported via TCP networking (there are other high-performance possibilities like shared performance, but networking is the default). So it doesn't matter at all where the application runs as long as the nodes can connect to each other. I guess that you want to test the application on your laptop, so the nodes are all running locally and can easily connect to each other via the loopback network.
I am not quite sure if I do understand your question, but a laptop is a computer just like any other. Providing you have set up your MPI libs correctly and set your paths, you can, of course, use MPI routines on your laptop.
As far as I am concerned, I use Debian Linux (http://www.debian.org) for all my parallel stuff. I have written a little article dealing with HowTo get MPI run on debian machines. You may want to refer to it.
Related
I have Fedora installed on my PC and I have a Friendly ARM Mini2440 board. I have successfully installed Linux kernel and everything is working. Now I have some image processing program, which I want to run on the board without OS. The only process running on board should be my program. And in that program how can I access the on board camera to take image from, and serial port to send output to the PC.
You're talking about what is often called a bare-metal environment. Google can help you, for example here. In a bare-metal environment you have to have a good understanding of your hardware because you have to take care of a lot of things that the OS normally handles.
I've been working (off and on) on bare-metal support for my ELLCC cross development tool-chain. I have the ARM implementation pretty far along but there is still quite a bit of work to do. I have written about some of my experiences on my blog.
First off, you have to get your program started. You'll need to write some start-up code, usually in assembly, to handle the initialization of the processor as it comes out of reset (or is powered on). The start-up code then typically passes control to code written in C that ultimately directly or indirectly calls your main() function. Getting to main() is a huge step in your bare-metal adventure!
Next, you need to decide how to support your hardware's I/O devices which in your case include the camera and serial port. How much of the standard C (or C++) library does your image processing require? You might need to add some support for functions like printf() or malloc() that normally need some kind of OS support. A simple "hello world" would be a good thing to try next.
ELLCC has examples of various levels of ARM bare-metal in the examples directory. They range from a simple main() up to and including MMU and TCP/IP support. The source for all of it can be browsed here.
I started writing this before I left for work this morning and didn't have time to finish. Both dwelch and Clifford had good suggestions. A bootloader might make your job a lot simpler and documentation on your hardware is crucial.
First you must realise that without an OS, you are responsible for bringing the board up from reset including configuring the PLL and SDRAM, and also for the driver code for every device on the board you wish to use. To do that required adequate documentation of the board and it devices.
It is possible that you can use the existing bootloader to configure the core and SDRAM, but that may not meet your requirement for the only process running on the board should be your image processing program.
Additionally you will need some means of loading and bootstrapping; again the existing Linux bootstrapper may suit.
It is by no means straightforward and cannot really be described in detail here.
I am currently in the process of developing the OS for a consumer electronics product my company is developing. I have settled on freeRTOS as the backbone for our OS, and am working diligently to implement hardware functionality within the OS. However, I have run into an issue concerning running 3rd-party applications from within freeRTOS.
Originally I considered a task to be an application, where basically you had "myapplication.c" and "myapplication.h" containing all your applications necessary functions and the code would reside within the for(;;) loop within the task (acting as a main while loop). Then when the user decides to run that application, a function pointer is passed to a queue, that my app_launcher task then uses to create the new task using the 3rd-party task or application.
The problem with this approach however, is the OS will already be compiled and reside on the microcontroller, and applications with be installed and deleted as the user sees fit... So obviously applications need to be compiled and executable from the OS. On a standard unix machine, I would use something like fork, to select the executable and give it it's own process. However I cannot find a similar functionality within freeRTOS.. My other idea is approaching a scripting language for app development, but again I'm not sure on how to launch those applications...
So the question is, how do I get freeRTOS to run applications from 3rd party developers that aren't already baked into the OS?
FreeRTOS (and most RTOSes for that matter) do not work like general purpose operating systems (GPOS), they are not generally designed to dynamically load and execute arbitrary user supplied applications. In most case you use an RTOS because you require hard real-time response and the execution of third-party code could compromise that.
Most RTOSes (FreeRTOS included) are no more that static-link libraries, where your entire embedded application is statically linked with the RTOS and executes as a single multi-threaded program.
Again many RTOSes (like FreeRTOS) are not operating systems in the same sense as a GPOS such as Linux. Typically the RTOS services available are the real-time scheduler, inter-process communication (IPC), thread-synchronisation, and timers. Middle-ware such as a file system, and network stack for example are either optional extensions or must be integrated from third-party code.
One problem you will have with FreeRTOS trying to achieve your aim is that a "task" is analogous to a "thread" rather than a "process" in the sense of a GPOS process model. A task typically operates in the same memory space as other tasks with no memory protection between tasks. Tasks are not separate programs, but threads within a single application.
If your target has no MMU then memory protection may be limited in any case, but you may still want third-party applications to be conceptually independent from the OS. If your processor does not have an MMU, then running arbitrary third-party dynamically loaded code may be a problem for system integrity, safety and security. Even with an MMU a simple RTOS kernel such as FreeRTOS won't use it.
Operating systems with real-time scheduling that can load and run application code dynamically as separate processes include:
Windows Embedded Compact (formerly Windows CE)
QNX Neutrino
OS-9
Also VxWorks has the ability to load partially linked object code and dynamically link it to the already loaded code. This is not the same at a process model, but is more akin to a dynamic-link library. What makes it worth mentioning in this context is that the VxWorks shell can invoke any function with external linkage by name. So you can load an object file implementing a function and then run that function. You could in principle implement the same functionality on FreeRTOS, but it is non trivial. The shell is one thing, but dynamic loading and linking requires the application symbol table to be target resident.
If you don't need hard real-time (or your real-time requirements are "soft") and your target has sufficient resources, you may be better served deploying Linux or uClinux which are increasingly used in embedded systems.
If the code your end-users need to run are tightly related to the purpose of your device rather than "general purpose" in nature then another possibility for allowing end-users to run code is to integrate a scripting language interpreter such as Lua. In this case you would simply load the script from a file system and pass it to the script interpreter. For more general purpose requirements a Java VM may be a possibility.
Due to request, here is the work around I found to my problem. The issue was launching other applications from freeRTOS. This was accomplished by utilizing the "System()" function in the newlib library. Thus, I can place an application in flash until it's needed, then launch it using the newlib functions provided. This also allows me to launch programs dynamically, without hard coding the code or name of the application, I just need to provide System() with a string, pointing to the app's location in memory.
I'm interested in studying the 9P FS, currently been reading the source available from these implementations: http://9p.cat-v.org/implementations
Is 9P obsolete? Are you using it for some application?
(also I've found this, some perfomance test between 9P and NFS: http://graverobbers.blogspot.com/2007/08/v9fs-performance-versus-nfs.html)
No, 9P isn't obsolete; I don't know of a protocol that does what it does and is clean and well defined enough to be implemented correctly in almost any language that exists.
9P is used in a variety of systems. A couple of recent uses in arm-js (an ARM emulator) and 9webdraw (a GSoC project that implements the Plan 9 /dev/draw). Both are HTML5 Javascript implementations.
Just to add a bit, both the Linux client implementation and several servers are under active development, so I'd say that's a pretty clear sign that folks still have use for it. One of the areas its seen heavy use more recently is the virtio-9P (aka virtfs) which is part of qemu/kvm and can be used for direct guest to host file access. It's also been used in several experimental operating systems projects (Libra, PROSE, FusedOS) and incorporated into other operating systems (BSD, MacOSX, Windows, Linux) and hypervisors (in addition to the KVM instance above, its also been incorporated in various ways into Xen). 9P is actually being used in supercomputing deployments (both for Plan 9 and Linux, see the diod project on Sourceforge).
I think the reason is that the protocol is quite simple, so implementations also tend to be quite simple and easy to integrate elsewhere (there are several applications both inside and outside the Plan 9 world which use 9P as an interface to the application, in much the same way that some web developers use RESTful interfaces).
The protocol has a couple of different variations including the 9P.L variant which was developed specifically to match the Linux VFS API better. It adds a bit of complexity to the protocol in the addition of operations, but removes some of the complexity of mapping Linux VFS API -> 9P and vice versa.
It is used in Erlang-on-Xen both as a storage protocol for goofs http://erlangonxen.org/blog/goofs-simple-filesystem
It is the way erlang on xen instances in other ways too, see here:
http://erlangonxen.org/more/9p2000e
Also, it's used by libvirt stuff with QEMU.
http://wiki.qemu.org/Documentation/9psetup
9p, to me, is like the Scheme of network protocols. For the most part, it is very simple, but people see need to extend it to fit their environments. Luckily this is done in ways that are often backwards compatible.
In addition to everything mentioned in the other answers, Microsoft is using 9P as part of their Windows Subsystem for Linux.
They add a 9P server to each Linux distribution that is running as a guest, so that Windows can mount the Linux filesystem over 9P, and Windows processes can transparently access the files on Linux's ext4 partition.
Next term, I'll need to write a basic operating system for Motorola 68K processor as part of a course lab material.
Is there a Linux emulator of a basic hardware setup with that processor? So my partners and I can debug quicker on our computers instead of physically restarting the board and stuff.
Is it possible to apply test-driven development technique to OS development? Code will be mostly assembly and C. What will be the main difficulties with trying to test-drive this? Any advice on how to do it?
I would recommend developing an operating system for the classic Amiga computers, which had different versions of the 68000 processor. Since the Amiga computer is a complete computer and is extremely well documented, I thought this would be a good exercise.
There is an emulator for it called UAE (and Win-UAE) which is very exact and
can be configured with different kinds of processors (68000 - 68060) and other capabilities. Normally, you would also need to acquire ROMs for it, but since you are developing an operating system yourself, this is not necessary.
Tools you will need is either Cygwin (for developing under Windows) or a Linux computer. Then you will need cross compilers. This includes both a C compiler and an assembler. Here is a template for creating a simple ROM which changes screen color and flicks the power LED. It will create a file 'kick.rom' which UAE then searches for in the current directory.
Reference on the 68000 instruction set can be found at the links below. Be aware that different assembler programs may use slightly different syntax and instruction set.
If you need to demo the operating system on real hardware, there are modern Amiga clones sold on Ebay and other places. Search for "Minimig".
Update:
Nowadays AROS also runs on UAE as well as physical Amigas.
Refs:
[UAE]
[WinUAE]
[Cygwin]
[Cross Compilers]
[68000 reference]
I would suggest QEMU for m68k emulation.
(The system emulator you want in QEMU is "Coldfire" - that's what Freescale calls the successor to the m68k architecture).
You certainly can tdd this project. First off decouple all accesses to the hardware with simple routine calls, e.g. getch() and printf, then you can provide simple mocks that provide test input and check output. You can then write well over 90% of the project on a PC using gcc, msdev or xcode. Once you have got some confidence in the decoupling routines you will need very little access to the hardware, and only then to occasionally check that your mocks are acting as you expect.
Keep to C until you find a particular bottle neck, and only then resort to assembler.
There are a few new projects that use hardware simulated 68000 cpus, the C-One project, the Minimig (Mini Amiga) project and the Natami (Native Amiga) project - they are new 68k compatible Amiga systems.
C One, reconfigurable computer, Minimig, in development, prototypes done: FPGA Arcade and Natami.
The Easy68k http://www.easy68k.com simulator might help you.
The uClinux project started on a m68k board. They may have the tools you need...
What things should be kept most in mind when writing cross-platform applications in C? Targeted platforms: 32-bit Intel based PC, Mac, and Linux. I'm especially looking for the type of versatility that Jungle Disk has in their USB desktop edition ( http://www.jungledisk.com/desktop/download.aspx )
What are tips and "gotchas" for this type of development?
I maintained for a number of years an ANSI C networking library that was ported to close to 30 different OS's and compilers. The library didn't have any GUI components, which made it easier. We ended up abstracting out into dedicated source files any routine that was not consistent across platforms, and used #defines where appropriate in those source files. This kept the code that was adjusted per platform isolated away from the main business logic of the library. We also made extensive use of typedefs and our own dedicated types so that we could easily change them per platform if needed. This made the port to 64-bit platforms fairly easy.
If you are looking to have GUI components, I would suggest looking at GUI toolkits such as WxWindows or Qt (which are both C++ libraries).
Try to avoid platform-dependent #ifdefs, as they tend to grow exponentially when you add new platforms. Instead, try to organize your source files as a tree with platform-independent code at the root, and platform-dependent code on the "leaves". There is a nice book on the subject, Multi-Platform Code Management. Sample code in it may look obsolete, but ideas described in the book are still brilliantly vital.
Further to Kyle's answer, I would strongly recommend against trying to use the Posix subsystem in Windows. It's implemented to an absolute bare minimum level such that Microsoft can claim "Posix support" on a feature sheet tick box. Perhaps somebody out there actually uses it, but I've never encountered it in real life.
One can certainly write cross-platform C code, you just have to be aware of the differences between platforms, and test, test, test. Unit tests and a CI (continuous integration) solution will go a long way toward making sure your program works across all your target platforms.
A good approach is to isolate the system-dependent stuff in one or a few modules at most. Provide a system-independent interface from that module. Then build everything else on top of that module, so it doesn't depend on the system you're compiling for.
XVT have a cross platform GUI C API which is mature 15+ years and sits on top of the native windowing toollkits. See WWW.XVT.COM.
They support at least LINUX, Windows, and MAC.
Try to write as much as you can with POSIX. Mac and Linux support POSIX natively and Windows has a system that can run it (as far as I know - I've never actually used it). If your app is graphical, both Mac and Linux support X11 libraries (Linux natively, Mac through X11.app) and there are numerous ways of getting X11 apps to run on Windows.
However, if you're looking for true multi-platform deployment, you should probably switch to a language like Java or Python that's capable of running the same program on multiple systems with little or no change.
Edit: I just downloaded the application and looked at the files. It does appear to have binaries for all 3 platforms in one directory. If your concern is in how to write apps that can be moved from machine to machine without losing settings, you should probably write all your configuration to a file in the same directory as the executable and not touch the Windows registry or create any dot directories in the home folder of the user that's running the program on Linux or Mac. And as far as creating a cross-distribution Linux binary, 32-bit POSIX/X11 would probably be the safest bet. I'm not sure what JungleDisk uses as I'm currently on a Mac.
There do exist quite few portable libraries just examples I've worked within the past
1) glib and gtk+
2) libcurl
3) libapr
Those cover nearly every platform and so they are extremly useful tool.
Posix is fine on Unices but well I doubt it's that great on windows, besides we do not have any stuff for portable GUIs there.
I also second the recommendation to separate code for different platforms into different modules/trees instead of ifdefs.
Also I recommend to check beforehand what are the differences in you platforms and how you could abstract them. E.g. this is some OS related stuff (e.g. the annoying CR,CRLF,LF in text files), or hardware stuff. E.g. the previous mentioned posix compability doesnt stop you from
int c;
fread(&c, sizeof(int), 1, file);
But on different hardware platforms the internal memory layout can be complete different (endianess), forcing you to use conversion functions on some of the target platforms.
You can use NAppGUI for both console and desktop apps. The SDK uses ANSI-C and your code will work on Windows/macOS/Linux.
https://www.nappgui.com
It's free and OpenSource.