I am trying to upload data to CKAN, and I can do that for smaller files (I have uploaded 4 kB successfully), however for bigger files (with a file of 18 MB I already got this error), I get Error 500 An internal server error occurred.
In the command prompt where I am running CKAN I get
Error - <type 'exceptions.WindowsError'>: [Error 32] The file is already being used by another process: u'C:\\src\\ckan\\ckan\\resources\\a3d\\19a\\ba-7f3f-42fc-
a02e-09f50aae0924~'
URL: http://localhost:5000/dataset/new_resource/test1
I don't know what that file is, but I am pretty sure this error is the reason why I can't upload larger files, as it is the only error I get.
Important to say that I can successfully add resources from URL and from small files, but when trying with larger files, I get this error.
Does anyone have any idea on what could be wrong here?
Many thanks!
I can't explain that Windows-Error, but generally CKAN has an upload size limit of 10MB for resources by default. You can raise that in your ini with ckan.max_resource_size = XX, for example ckan.max_resource_size = 100 (which means = 100MB).
Related
After booting SWUpdate yocto-generated image for the first time, executing swupdate results in error message:
Error parsing configuration file: 'globals' section missing, exiting.
I tried to strictly follow SWUpdate's documentation, but it gets short when it comes to yocto integration. I'm using meta-swupdate, meta-swupdate-boards, and meta-openembedded layers together with poky example repository all at Kirkstone tag, building via bitbake update-image and having modyfied local.conf as:
MACHINE ??= "raspberrypi4-64"
ENABLE_UART = "1"
RPI_USE_U_BOOT = "1"
IMAGE_FSTYPES = "wic ext4.gz"
PREFERRED_PROVIDER_u-boot-fw-utils = "libubootenv"
IMAGE_INSTALL:append = " swupdate"
Is there anything else I need to modify to generate the configuration file and be able to run SWUpdate binary properly?
Side question: In the documentation, it's recommended to append swupdate-www to achieve a better web server. However, if I append it, there is no swupdate-www binary inside the `/usr/bin' directory.
As with other recipes folders the recipes-support/swupdate/swupdate/raspberrypi4-64 folder was missing inside the meta-swupdate-boards layer. Therefore, an empty config file was always generated. After adding this folder and all related files, strongly inspired by raspberrypi3 folder, the error was gone and swupdate -h provided the expected output.
There was also one new error during build process thrown by yocto. It was related to missing systemd requirement and was solved by adding:
DISTRO_FEATURES_append = " systemd"
to local.conf
I'm running into a very peculiar and undocumented issue with a GAE Managed VM and Task Queues. I understand that the Managed VM service is in beta, so this question may not be relevant forever, but it's definitely causing me lots of headache now.
The main symptom of the issue is that, in certain (not completely known to me) circumstances, I'm seeing the following error/traceback:
File "/home/vmagent/my_app/some_file.py", line 265, in some_ndb_tasklet
res = yield some_task.add_async('some-task-queue-name')
File "/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/tasklets.py", line 472, in _on_rpc_completion
result = rpc.get_result()
File "/home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/home/vmagent/python_vm_runtime/google/appengine/api/taskqueue/taskqueue.py", line 1948, in ResultHook
rpc.check_success()
File "/home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success
self.__rpc.CheckSuccess()
File "/home/vmagent/python_vm_runtime/google/appengine/ext/vmruntime/vmstub.py", line 312, in _WaitImpl
raise self._ErrorException(*_DEFAULT_EXCEPTION)
RPCFailedError: The remote RPC to the application server failed for call taskqueue.BulkAdd().
I've gone through my local App Engine SDK to trace this through, and I can get up to the last line of the trace, but google/appengine/ext/vmruntime/ doesn't exist on my machine at all, so I have no idea what's happening in vmstub.py. From looking at the local code, some_task.add_async('the-queue') is spinning up an RPC and waiting for it to finish, but this error is not what the except apiproxy_errors.ApplicationError, e: at line 1949 of taskqueue.py is expecting...
The code that's generating the error looks something like this:
#ndb.tasklet
def kickoff_tasks(batch_of_payloads):
for task_payload in batch_of_payloads:
# task_payload is a dict
task = taskqueue.Task(
url='/the/handler/url',
params=payload)
res = yield task.add_async('some-valid-task-queue-name')
Other things worth noting:
this code itself is running in a task handler kicked off by another task.
I first saw this error before implementing this sort of batching, and assumed the issue was because I had added too many tasks from within a task handler.
In some cases, I can run this successfully with a batch size of 100, but in others, it fails consistently (depending on the data in the payloads) at 100, and sometimes succeeds at batch sizes of 50.
The task payloads themselves include batches of items, and are tuned to be just small enough to fit in a task. App Engine advertises a maximum task size of 100KB, so I'm keeping the payloads to under 90,000 bytes right now. Lowering the size even more doesn't seem to help any.
I've also tried implementing an exponential backoff to retry the kickoff_tasks method when this error appears, but it seems that once the error is raised, I can't add any other tasks at all from within the same handler (i.e. I can't kickoff a "continue from where you left off" task, I just have to let this one fail and restart itself)
So, my question is, what is actually causing this error? How can I avoid it, or fix this so that I'm handling it correctly?
This is a known issue that is being worked on. There are actually two issues - the RPC failure itself and the lack of handling of the RPCFailedError exception by the SDK.
There is some public discussion of the issue here.
If you're using App Engine Flexible and the python-compat-multicore image, a new bug popped up related to App Engine using a newer version of the requests library that broke the communication between App Engine Flexible and the datastore. You can fix this error by monkey patching the library in your appengine_config.py file.
Add the following code to appengine_config.py:
try:
import appengine.ext.vmruntime.vmstub as vmstub
except ImportError:
pass
else:
if isinstance(vmstub.DEFAULT_TIMEOUT, (int, long)):
# Newer requests libraries do not accept integers as header values.
# Be sure to convert the header value before sending.
# See Support Case ID 11235929.
vmstub.DEFAULT_TIMEOUT = bytes(vmstub.DEFAULT_TIMEOUT)
Note that if you do not have an appengine_config.py file, you can just create it in your base project directory (wherever you put your app.yaml file). This file gets run during App Engine startup..
I am trying to run ADNI .mnc image in MATLAB
I added folder emma-master, niak-0.7.1-ammo, mia and niak-0.7.1-ammo to my path. All these folders are located in
D:\EMINA BURCH\PhD Thesis\MATLAB Packages
But when I want to open ._bq_t_15T.mnc located also in D:\EMINA BURCH\PhD Thesis\MATLAB Packages
that is h = openimage('._bq_n_15T.mnc')
I get the following error
Error using miinquire (line 145)
Error getting image dimensions from file D:\EMINA BURCH\PhD Thesis\MATLAB Packages._bq_n_15T.mnc
Error in openimage (line 173)
DimSizes = miinquire (filename, 'imagesize');
When I enter debug mode in minquire function after the line
[stat,out] = system(['mincinfo -vardims image ' minc_file]);
I get stat = 1 and out = 'mincinfo' is not recognized as an internal or external command, operable program or batch file.
May You, please help me with this issue.
system isn't able to automatically recognise the different requirements that might be needed to run the same system command on different operating systems. Using commands like ispc, isunix, ismac and computer, you can automatically check and call different versions of the commands as appropriate:
if ispc
[stat,out] = system(['mincinfo.exe -vardims image ' minc_file]);
else
[stat,out] = system(['mincinfo -vardims image ' minc_file]);
end
Of course, you also have to have the appropriate program on your MATLAB path for this to work.
I am using OpenSearchServer v1.2.4 rc3.
In the first few days it's working fine.
But when its Index size reached 1.0GB I got this error
"Error (java.lang.NullPointerException)"
when I start my crawler. The crawler works fine for some time and then stops with this error
"Error (java.lang.NullPointerException)".
What's wrong?
Depending of the size of your index, a memory parameter must be added. By default, OpenSearchServer is setup to run on small server with the default RAM value provided by the Java Virtual Machine (from 64MB to 512MB only).
For large indexes, you must set up a higher value. On a Unix/Linux server, just create an /etc/opensearchserver file with the following content:
CATALINA_OPTS="-Xms2G -Xmx2G -server"
export CATALINA_OPTS
On a Windows server, edit the start.bat files. Add the following line just after :okExec
set CATALINA_OPTS="-Xms2G -Xmx2G -server"
Replace 2G (which mean 2 GB) by the size of the memory you want to allocate to OpenSearchServer.
In a 32 bits version, the memory is limited to 2.5GB. You can use more memory with a 64 bits operating system using the following lines (on Unix/Linux):
CATALINA_OPTS="-Xms12G -Xmx12G -d64 -server"
for Window 64bits:
set CATALINA_OPTS="-Xms12G -Xmx12G -d64 -server"
After restarting OpenSearchServer, just check in the Runtime tab panel that you have the correct size of memory available.
Regarding the error details, it is more useful to have the full stack trace. You can find it in the log file (data/logs/oss.log), or in the Runtime/Logs tab panel.
This is a cross post from Perl Monks and Mahalo answers, where I have not received a satisfactory response yet. Thanks for your time and spirit:
Why do I get this error message from perl:
Can't call method "scaleY" on an undefined value at C:/strawberry/perl +/site/lib/ Image/Seek.pm line 137?
I am getting the error in the title when calling the Image::Seek module from my script. My script is basically a rehash of the module's suggested code.
Here's the error again:
Can't call method "scaleY" on an undefined value at C:/strawberry/perl +/site/lib/ Image/Seek.pm line 137.
Here's my code:
#!/usr/local/bin/perl
use Imager;
use Image::Seek qw(loaddb add_image query_id savedb);
loaddb("haar.db");
my $img = Imager->new("photo-1.jpg")
or die Imager->errstr;
# my $img = Imager->new();
# $img->open(file => "photo-1.jpg")or die Imager->errstr;
add_image($img, 1);
savedb("haar.db");
Here's the section of the Image::Seek module causing the issue:
sub add_image_imager {
my ($img, $id) = #_;
my ($reds, $blues, $greens);
require Imager;
my $thumb = $img->scaleX(pixels => 128)->scaleY(pixels => 128);
for my $y (0..127) {
my #cols = $thumb->getscanline(y => $y);
for (#cols) {
my ($r, $g, $b) = $_->rgba;
$reds .= chr($r); $blues .= chr($b); $greens .= chr($g);
}
}
addImage($id, $reds, $greens, $blues); }
Line 137 is:
my $thumb = $img->scaleX(pixels => 128)->scaleY(pixels => 128);
If I remove
->scaleY(pixels => 128)
then line 129:
my #cols = $thumb->getscanline(y => $y);
gives me essentially the same error.
At this point I'm just trying to add one image to the database. There is an image in the directory where I'm running the script to add the image, named "photo-216.jpg". If I change the name to "photo-1.jpg" or "photo-0.jpg" and change the corresponding "add_image" and "query_id" to respectively 1 or 0, it's the same result.
I do have a database that is 385 KB big that comes from running makedb.pl below, but it is filled with null characters. I renamed this "haar.db". This is the database that gives me the error. If I recreate the haar.db file as an empty one, then the script hangs and after a couple of minutes, it give this different message:
"This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information."
If there is no "haar.db" the file still gives me the error in this post's title and unlike running makedb.pl, gives me no database named "haar.db".
By the way I get multiple examples of this post's title error also when trying to run this database filling script: http://www.drk7.jp/pub/imgseek/t/makedb.pl.txt/, which I was alluding to before. I obviously removed the .txt extension before trying it. The makedb.pl script is from this Japanese site: http://www.drk7.jp/MT/archives/001258.html.
If I run makedb.pl in a directory of 2423 scanned collectible postage stamps images, I get 362 instances of the error. The 2423 stamps is the number I have after removing the "small" thumbnail versions which I orignally thought might be causing the issue.
Could it be, that some of the images are less than 128 pixels and that is the issue? However if this is true why does the database get filled with null characters?...Unless they are not really null even though the editor I'm using, Notebook++, says they are.
Also note my images are of stamps which are only sometimes perfect squares. Otherwise, sometimes they are "landscape" sometimes "portrait". Maybe the issue is when the "landscape" scaled images get an X axis of 128 pixels and then their Y axis ends up less or much less. Could this be?
Thanks much
Update: Answer completely re-organized.
Image::Seek is not checking if
scaleX returned error. In your case, for some images, scaleX is failing.
You seem to know for which images scaleX is failing. So, leave your current
code aside, and put together a short test script:
#!/usr/bin/perl
use strict;
use warnings;
use Imager;
die "Specify image file name\n" unless #ARGV;
my ($imgfile) = #ARGV;
my $img = Imager->new;
$img->read( file => $imgfile )
or die "Cannot read '$imgfile': ", $img->errstr;
my $x_scaled = $img->scaleX( pixels => 128 )
or die 'scaleX failed: ', $img->errstr;
my $thumb = $x_scaled->scaleY( pixels => 128 )
or die 'scaleY failed: ', $x_scaled->errstr;
__END__
Running this test script, you got the error message:
Cannot read 'photo-1.jpg': format 'jpeg' not supported - formats bmp,
ico, pnm, raw, sgi, tga available for reading
indicating the underlying problem: When you installed Imager via Strawberry
Perl's cpan, the libraries for png, jpg etc were not installed. One
solution is to build those libraries with the gcc compiler provided with
Strawberry Perl.
First, you will need zlib.
C:\Temp\zlib-1.2.3> copy win32\Makefile.gcc Makefile
Set prefix = /strawberry/c/local in the Makefile. Compile. You may have to
manually copy the files zlib.h and zconf.h to
C:\strawberry\c\local\include and zlib1.dll, libz.a and libzdll.a to
C:\strawberry\c\local\lib (I don't know because I do not use Strawberry Perl very often and my Strawberry environment is very neglected.)
Then, get libpng. I used the source archive without config script.
C:\Temp\libpng-1.2.38> copy scripts\makefile.mingw Makefile
C:\Temp\libpng-1.2.38> make prefix=/strawberry/c/local ZLIBLIB=/strawberry/c/local/lib ZLIBINC=/strawberry/c/local/include
This built the PNG library. Again, you may have to manually copy the .dll,
.a and .h files to the appropriate directories. I did because of my less
than perfect Strawberry environment.
Finally, get the JPEG library.
C:\Temp\jpeg-7> copy Makefile.ansi Makefile
Make sure to edit this file and set CC=gcc. Customize jconfig.h according
to the instructions in jconfig.txt. I used jconfig.dj as a basis.
You might also want to set
CFLAGS= -O2
SYSDEPMEM= jmemansi.o
in Makefile, and
#define DEFAULT_MAX_MEM 4*1024*1024
in jconfig.h. After running make, again copy the files as needed (and as explained by install.txt).
Once the libraries are installed, you can
C:\Temp> SET IM_INCPATH=C:\strawberry\c\local\include
C:\Temp> SET IM_LIBPATH=C:\strawberry\c\local\lib
C:\Temp> cpan
cpan> force install Imager
which yields:
gif: includes not found - libraries not found
ungif: includes not found - libraries not found
jpeg: includes found - libraries found
png: includes found - libraries found
tiff: includes not found - libraries not found
freetype2: includes not found - libraries not found
freetype2: not available
T1-fonts: includes not found - libraries not found
TT-fonts: includes not found - libraries not found
w32: includes found - libraries found
If all of this is too much work, it is ... sigh I just realized the
binaries are available at GnuWin32.