I'm looking to see if there is a best practice for building a perl526 program using scl perl526.
I'll be using git clone to pull the source code in, and then building it with scl rh-perl526 enable bash
Would I git clone into /opt/rh/rh-perl526/root/opt or is it better placed elsewhere?
The program I'm building is https://github.com/Ensembl/ensembl-vep.git and it builds quite nicely, but has various optional things to add.
The end program needs to be accessible to all users once built.
Thank you
Bryan
Related
I am building a kiosk application using webkitgtk on the raspberry pi 4.
This application will not be connected to the internet and all the html,css, javascript for the UI are all located on the local filesystem.
I am using buildroot to setup the Linux system, starting with the pi 4 defconfig provided in buildroot.
I have enabled all the packages needed to get webkitgtk running.
Also, the kiosk application has been tested on my desktop, using the same software stack and it works
However, when i try to launch the application on the raspberry pi, a blank page pops up. I have played around with the WebKitWebSettings object associated with my WebKitWebView by enabling local file access. It still shows up a blank screen.
Also included in my pi4 application bundle is a simple gtk3+ application. This launches successfully!
I will really appreciate some pointers as to why this is happening as i have sort of reached a dead end
UPDATE
I enabled the MiniBrowser app that comes with the Webkitgtk package.
Entering the local url, The page does not load. It only gives me a message at the top saying "Successfully downloaded".
It seems to be treating my input as a download
UPDATE 2
After some more experimenting, i was finally able to get webkitgtk working on the pi 4.
The problem seems to originate from using the webkit_web_view_load_uri() api.
It does not seem to recognize my html document as a web page.
I got around it using the webkit_web_view_load_html() call. This included some hacks by first reading in the contents of the html doc into a character buffer, and passing it to webkit_web_view_load_html().
You also have to provide a base path to this function call to be able to resolve all the urls (scripts, css, images etc) in your html document.
Another problem i haven't been able to work around is, SVG images are not loading in webkitgtk. I have used jpg formats and they work. I suspect this my be due to a configuration switch in building webkigtk
It's hard for me to figure out what might be happening without having access to your environment and settings. My gut feeling is that pages are showing blank because perhaps some shared libraries are missing. You can check that with:
$ ldd WebKitBuild/GTK/Release/bin/MiniBrowser
I am using buildroot to setup the Linux system, starting with the pi 4 defconfig provided in buildroot.
There's a buildroot repository for building WPE for RPi. WPE (WebPlatform for Embeded) is like WebKitGTK but doesn't depend on GTK toolkit. Another important difference is that WPE runs natively on Wayland.
If you're interested in having a webapp embedded in a browser running in a device with limited capabilities, WPE is a better choice than WebKitGTK. The buildroot repo for building WPE for RPi is here:
https://github.com/WebPlatformForEmbedded/buildroot
There's is also this very interesting step-by-step guide on how to build WPE for RPi3:
https://samdecrock.medium.com/building-wpe-webkit-for-raspberry-pi-3-cdbd7b5cb362
I'm not sure whether the buildroot recipe would work for RPi4. It seems to work for all previous versions, so you might be stepping in new land if you try to build WPE on RPi4.
If you have an RPi3 available I'd try to build WPE for RPi3 first, and make sure that works. Then try for RPi4.
Can nix be used in a continuous-delivery workflow?
We're using semaphore as our continuous integration service, and now I'm looking into building packages upon a successful build. For this I'm considering using nix.
I don't know what would be the right way of setting up a continuous delivery pipeline with this package manager. It seems that such an automated process would involve:
Making a branch of the nixpkgs repository (in the CI server).
Updating the rev field of fetchFromGithub.
(automatically) submitting a pull-request.
But I don't know if this makes sense, and also I'm concerned that the continuous-delivery process involved a manual step (having an human aproving the pull-request).
Can nix be used in a continuous-delivery workflow?
Yes. It's typically done with Hydra, a CI system built with Nix. But, it may be possible to do this with Semaphore.
Semaphore CI provides build environments that are language specific, but... it's running Ubuntu, so theoretically you can do something like this:
Install Nix as if it were a dependency. See this article.
Add your Nix package, which I suppose you can do with Git. You don't really need to clone Nixpkgs.
Use nix-build to build your package. This will create a result symbolic link to the build output.
Deploy using git-deploy.
If you do something like this with your package you can call it directly from nix-build because you won't have to provide the package dependencies as arguments:
{ pkgs ? import <nixpkgs> {} }:
let
stdenv = pkgs.stdenv;
...
in
stdenv.mkDerivation {
..
}
Optimization
Installing Nix for every build is wasteful, but perhaps you can cache the Nix store. See this article.
My question should be very simple to answer for anyone not being a self-taught newbie like me...
On this page is a cheatsheet concerning a function to be used in GIS/DB environnement : http://www.bostongis.com/pgsql2shp_shp2pgsql_quickguide.bqg
I would like to create a script allowing users to just have to click on it to launch the process, given the proper datas. But I don't understand how to use this. It obviously doesn't work in a Python console, nor directly in the windows console. How is it supposed to work ? What language is this ?
Thanks
shp2pgsql is indeed a command line tool. It comes with your PostgreSQL/PostGIS installation (usually) and, if not accessible via PATH-variable, can (usually) be run from within the /bin-folder in your PostgreSQL-Installation. You can also always 'make' the programm from source in any location yourself, if needed.
EDIT:
One way to set up a script (independent of whether you use it within qgis own python environment or not) would be to use Pythons subprocess (or os.system) module (check related question here) to write to shell and execute shp2pgsql.
A slightly more sophisitcated solution to (batch) insert (multiple) shapefiles via script could be to implement ogr2ogr via gdal/ogr module within python (check this blog). That, however, would require a working installation of the gdal core library, and the respective Python bindings (at least to use outside of QGIS Python environment, where it is pre-installed AFAIK), which can be tiresome at times. Once installed correctly, it offers a powerful (I dare say almighty) toolset for geodata management and manipulation via Python, though.
Apart from that, the blog link I provided also states the implementation of a batch insert script/tool (which operates ogr2ogr) in qgis 2.8 toolbox...maybe that can help you, either with your work directly or (via sourcecode) to point you in the direction of creating your own tool.
Which configuration management tool is the best for FPGA designs, specifically Xilinx FPGA's programmed with VHDL and C for the embedded (microblaze) software?
There isn't a "best", but configuration control solutions that work for software will be OK for FPGAs - the flow is very similar. I use Subversion at work and git at home, and wrote a little on 'why' at my blog.
In other answers, binary files keep getting mentioned - the only binary files I deal with are compilation products (equivalent to software object and executables), so I don't keep them in the version control repository, I keep a zipfile for each release/tag that I create with all the important (and irritatingly slow to reproduce) ones in.
I don't think it much matters what revision control tool you use -- anything that you would consider good in general will probably be OK here. I personally use Git for a sizable Verilog + software project, and I'm quite happy with it.
What will bite you in the ass -- no matter what version control you use -- is this: The Xilinx tools don't generally respect a clean division between "input" and "output" or between (human edited) "source" and (opaque) "binary." Many of the tools like to store some state information, like a last-run time or a hash value, in their "input" files meaning that you'll get lots of false changes. Coregen does this to its .xco files, and project navigator (the main GUI) does this to its .xise files. Also, both tools have a habit of inserting or removing lines for default-valued parameters, seemingly at random.
The biggest issue I've encountered is the work-flow with Coregen: In many cases, at least one of the following is true:
You have to manually edit the HDL files produced by Coregen.
The parameters that went into Coregen are stored somewhere other than the .xco file (usually in what looks like an output file).
You have to copy-and-paste the output from Coregen into your top-level design.
This means that there is no single logical source/master location for your input to the core-generating process. So even if you have the .xco file under version control, there's no expectation that the design you're running corresponds to it. If you re-generate "the same" core from its nominal inputs, you probably won't get the right outputs. And don't even think about merging.
I suggest CM tools that support version labeling and binary files. Most Software CM applications are fine with ASCII text files. They may just store a "difference" file rather than the entire file for updates.
My recommendations: PVCS, ClearCase and Subversion. DO NOT USE Microsoft SourceSafe. I don't like it because it only supports one label per revision.
I've seen Perforce and Subversion used in a couple of FPGA-intensive companies.
We use Perforce, and its great. You can have your code that lives in Linux-land checked in side-by-side with your Specs and Docs that live in Windows-land. And you get branching, labels, etc.
I've seen everything from Clearcase to RCS used, and it is really all okay for this kind of thing. The important thing is to get a good set of check-in policies established for your group, and make sure they stick to it.
And have automated nightly regressions. That way, when someone breaks the rules, they can be identified and publicly shamed.
I have personally used Perforce, Subverion, git and ClearCase for FPGA projects. Since VHDL and C are just text files, any works fine. However be sure to capture the other project and contraint files and any libraries you use.
Also think about what to do with the outputs, e.g. log file and bitstreams. Both tend to be big and the bitstreams are binaries.
Previously I used Subversion but have switched to git two years ago. Git handles FPGA design files just as well as it handles every other text and binary file. Git is all you need for version controlling your files and artifacts.
For building the designs, I recommend just using a single ISE project called "ise" (living in a subdirectory called "ise/"). You can take a look at my (very modest) FPGA open-source project on github for the file layout. I don't bother storing the ISE files at all since they are easy to regenerate. The only things I save are the Verilog files and some ISIM waveform config files. In other projects that use coregen I save the coregen.cgp project file and all of the *.xco scripts for regenerating cores. Then I use a Makefile for actually running coregen on the *.xco files. There are a few other Xilinx-specific files you should version control too: *.ucf, *.coe, *.xcf, etc.
I experimented with using Makefiles and the Xilinx command-line tools but found that ISE did a much better job tracking dependencies and calling the tools with the right arguments. Just don't make the mistake of trying to version control your ise/ project files or you will go mad. Xilinx has something like 300 different file types which change every release. If you want to save a file, you can try the ISE project file itself with a .xise extension. Anything that is hard to recreate, like the golden bitfile that you know works and took 6 hours to build, you might want to copy that and configuration manage it explicitly.
I'm doing some Linux kernel development, and I'm trying to use Netbeans. Despite declared support for Make-based C projects, I cannot create a fully functional Netbeans project. This is despite compiling having Netbeans analyze a kernel binary that was compiled with full debugging information. Problems include:
files are wrongly excluded: Some files are incorrectly greyed out in the project, which means Netbeans does not believe they should be included in the project, when in fact they are compiled into the kernel. The main problem is that Netbeans will miss any definitions that exist in these files, such as data structures and functions, but also miss macro definitions.
cannot find definitions: Pretty self-explanatory - often times, Netbeans cannot find the definition of something. This is partly a result of the above problem.
can't find header files: self-explanatory
I'm wondering if anyone has had success with setting up Netbeans for Linux kernel development, and if so, what settings they used. Ultimately, I'm looking for Netbeans to be able to either parse the Makefile (preferred) or extract the debug information from the binary (less desirable, since this can significantly slow down compilation), and automatically determine which files are actually compiled and which macros are actually defined. Then, based on this, I would like to be able to find the definitions of any data structure, variable, function, etc. and have complete auto-completion.
Let me preface this question with some points:
I'm not interested in solutions involving Vim/Emacs. I know some people like them, but I'm not one of them.
As the title suggest, I would be also happy to know how to set-up Eclipse to do what I need
While I would prefer perfect coverage, something that only misses one in a million definitions is obviously fine
SO's useful "Related Questions" feature has informed me that the following question is related: https://stackoverflow.com/questions/149321/what-ide-would-be-good-for-linux-kernel-driver-development. Upon reading it, the question is more of a comparison between IDE's, whereas I'm looking for how to set-up a particular IDE. Even so, the user Wade Mealing seems to have some expertise in working with Eclipse on this kind of development, so I would certainly appreciate his (and of course all of your) answers.
Cheers
Eclipse seems to be pretty popular for Linux kernel development:
http://cdtdoug.blogspot.com/2008/12/linux-kernel-debugging-with-cdt.html
http://jakob.engbloms.se/archives/338
http://revver.com/video/606464/debugging-the-linux-kernel-using-eclipsecdt-and-qemu/
I previously wrote up an answer. Now I come up with all the details of the solution and would like to share it. Unfortunately stackoverflow does not allow me to edit the previous answer. So I write it up in this new answer.
It involves a few steps.
[1] The first step is to modify linux scripts to leave dep files in. By default after using them in the build, those dep files are removed. Those dep files contains exact dependency information about which other files a C file depends. We need them to create a list of all the files involved in a build. Thus, modify files under linux-x.y.z/scripts to make them not to remove the dep files like this:
linux-3.1.2/scripts
Kbuild.include: echo do_not_rm1 rm -f $(depfile);
Makefile.build: echo do_not_rm2 rm -f $(depfile);
The other steps are detailed in my github code project file https://github.com/minghuascode/Nbk/blob/master/note-nbkparse. Roughly you do:
[2] Configure with your method of configuration, but be sure use "O=" option to build the obj files into a separate directory.
[3] Then use the same "O=" option and "V=1" option to build linux, and save make output into a file.
[4] Run my nbkparse script from the above github project. It does:
[4.1] Read in the make log file, and the dep files. Generate a mirroring command.
[4.2] Run the mirroring command to hard-link the relevant source files into a separate tree, and generate a make-log file for NetBeans to use.
Now create a NetBeans C project using the mirrored source tree and the generated log file. NetBeans should be able to resolve all the kernel symbols. And you will only see the files involved in the build.
The Eclipse wiki has a page about this: HowTo use the CDT to navigate Linux kernel source
I have been doing some embedded linux development. Including kernel module development and have imported the entire linux kernel source code into Eclipse, as a separate project. I have been building the kernel itself outside of Eclipse(so far), but I don't any reason why I shouldn't be able to set up the build environment within Eclipse to build the kernel. For my projects, as long as I setup the PATH properties to point to the appropriate linux source include directories, it seems to be pretty good about name completion for struct fields, etc.
I can't really comment, on if it is picking up the correct defines and not greying out the correspond sections, as I haven't really paid to much attention to the files within the kernel itself.(so far)
I was also wondering about using Netbeans as a linux 'C' IDE, as I do prefer Netbean's for Java GUI development.
I think this would work (done each step for various projects):
[1] Modify kernel build scripts to leave .d files. By default they are removed.
[2] Log the build process to a file.
[3] Write a script to parse the build log.
[3.1] From the build log, you know every .c files.
[3.2] From the .c file, you know which is the corresponding .d file.
[3.3] Look into .d files to find out all the included .h files.
[3.4] Form a complete .c and .h file list.
[4] Now create a new dir, and use "ln -s" or "ln" to pick files of interest.
Now, create a Netbeans project for existing source code in the [4].
Configure code assistance to use make-log file. You should see
exactly the effective source code as when you build it at [2].
Some explanations to the above steps:
At [2], do a real build so the log file contains the exact files and flags of interest.
Later netbeans will be able to use the exact flags to parse.
At [4], pick only the files you want to see. Incorporating the whole kernel tree into netbeans will be unpractical.
There is a trick to parsing .d files: Many of the depended items are not real paths to a .h file, they are a modified entry for part of the linux config sections in the auto config file. You may need to reverse the modification to figure out which is the real header file.
Actually there is a topic on netbeans site. This is the discussion url: http://forums.netbeans.org/ntopic3075.html . And there is a wiki page linked from the discussion: wiki.netbeans.org/CNDLinuxKernel . Basically it asks you to prefix make with CFLAGS="-g3 -gdwarf-2" .
I found this link very helpful in setting up proper indexing in Eclipse. It requires running a script to alter Eclipse environment to match your kernel options, in my case
$ autoconf-to-eclipse.py ./include/generated/autoconf.h .
An illustrated guide to indexing the linux kernel in eclipse