SHFileOperation: randomly raises exceptions when deleting files - c

I am using SHFileOperation() to delete directories from a specific path. It is done in multiple threads and the directory that's deleted is always different.
From time to time, it throws exceptions:
Exception thrown at 0x00007FF8AF5D9D2A (ntdll.dll) in del.exe:
0xC0000008: An invalid handle was specified
and this one:
Exception thrown at 0x00007FF8ACC90A36 (shell32.dll) in del.exe:
0xC0000005: Access violation reading location 0x0000000000000001.
modules:
shell32.dll 00007FF8ACBD0000-00007FF8AE0D8000
ntdll.dll 00007FF8AF530000-00007FF8AF701000
This is the code:
SHFILEOPSTRUCTW tFileOptions = { 0 };
/* Initialize the file options structure for the deletion process */
tFileOptions.pFrom = pwstrPath;
tFileOptions.wFunc = FO_DELETE;
tFileOptions.fFlags = FOF_NOCONFIRMATION | FOF_SILENT | FOF_NOERRORUI;
/* Execute the deletions with the Shell operation */
iResult = SHFileOperationW(&tFileOptions);
if (0 != iResult)
{
printf("WTF\n");
goto lbl_cleanup;
}
SHChangeNotify(SHCNE_RMDIR, SHCNF_PATHW, pwstrPath, NULL);
The pwstrPath has a double null terminator at the end.
What can be the reason for these exceptions?
EDIT
Stack trace:

from stack trace (even without pdb symbols - with it will be much more better ) visible that exception not inside windows shell itself but in third party product - dragext64.dll (this is not native windows image) wich is implement Copy Hook Handler - i advise uninstall this or disable via registry key
HKEY_CLASSES_ROOT
Directory
shellex
CopyHookHandlers
MyCopyHandler
(Default) = {MyCopyHandler CLSID GUID}
and test after this. think exceptions must go away.
also look like some another shell extensions have bugs here - search in google SHELL32_CallFileCopyHooks. for example bug TortoiseGit.dll - note here in stack trace shell32.dll!SHELL32_CallFileCopyHooks()
so all this bugs inside implementation of ICopyHook::CopyCallback method

Related

SWUpdate on RPi4 via yocto - error parsing configuration file

After booting SWUpdate yocto-generated image for the first time, executing swupdate results in error message:
Error parsing configuration file: 'globals' section missing, exiting.
I tried to strictly follow SWUpdate's documentation, but it gets short when it comes to yocto integration. I'm using meta-swupdate, meta-swupdate-boards, and meta-openembedded layers together with poky example repository all at Kirkstone tag, building via bitbake update-image and having modyfied local.conf as:
MACHINE ??= "raspberrypi4-64"
ENABLE_UART = "1"
RPI_USE_U_BOOT = "1"
IMAGE_FSTYPES = "wic ext4.gz"
PREFERRED_PROVIDER_u-boot-fw-utils = "libubootenv"
IMAGE_INSTALL:append = " swupdate"
Is there anything else I need to modify to generate the configuration file and be able to run SWUpdate binary properly?
Side question: In the documentation, it's recommended to append swupdate-www to achieve a better web server. However, if I append it, there is no swupdate-www binary inside the `/usr/bin' directory.
As with other recipes folders the recipes-support/swupdate/swupdate/raspberrypi4-64 folder was missing inside the meta-swupdate-boards layer. Therefore, an empty config file was always generated. After adding this folder and all related files, strongly inspired by raspberrypi3 folder, the error was gone and swupdate -h provided the expected output.
There was also one new error during build process thrown by yocto. It was related to missing systemd requirement and was solved by adding:
DISTRO_FEATURES_append = " systemd"
to local.conf

what does gitolite setup fix?

gitolite info didn't work, adding keys turned them into a no access key and did NOT create a corresponding entry in auth-keys file.
To fix this run gitolite setup on gitolite server
Question: what could have landed me in that mess?
And what does gitolite setup do when invoked for the n-th time (it's no longer setting things up, according to the docs it fixes hooks, but I wonder what the use case would be and which was mine)?
More details on gitolite info
gitolite info command is invoked like so:
> ssh git-user#ser-git
PTY allocation request failed on channel 0
hello git-admin, this is ...#... running gitolite3 3.6.7-2 (Debian) on git 2.17.1
R W some-repository
R W gitolite-admin
R W testing
Connection to ser-git closed.
Bad output is: FATAL: unknown git/gitolite command: 'info'
More details: keys without access.
gitolite sshkeys-lint was showing keys with (no access), now those keys have access as I set them (now meaning after gitolite setup).
ssh-keygen -lf /home/repo/.ssh/authorized_keys | wc -l (or without piped part, regardless) number of keys and their names indicated I didn't have the newest one added.
Similar question that did not work for me: keydir entries not propagating to authorized_keys
Docs pretty much had the answer once I dug deeper, I guess. Which is fairly nice of #sitaramc.
Without options, 'gitolite setup' is a general "fix up everything" command
(for example, if you brought in repos from outside, or someone messed
around with the hooks, or you made an rc file change that affects access
rules, etc.)
Symptoms keys stopped propagating and error FATAL: unknown git/gitolite command: 'info' on ssh git-user#ser-git. Fix was to run gitolite setup. So onto first question, the title one:
what does gitolite setup fix?
gitolite setup is implemented here
my Perl is rather weak, but there's a setup function in line 56. It calls args (which parses options, so here it had nothing to parse), then unless h_only (hooks only arg for setup), which wasn't used, so we skip compile and POST_COMPILE trigger and go for the hooks.
sub setup {
my ( $admin, $pubkey, $h_only, $message ) = args();
unless ($h_only) {
setup_glrc();
setup_gladmin( $admin, $pubkey, $message );
_system("gitolite compile");
_system("gitolite trigger POST_COMPILE");
}
hook_repos(); # all of them, just to be sure
}
package Gitolite::conf::store has hook_repos(), line 228: we change the dir to repo base dir (as per config file), and for each phy_repo we do hook_1(phy_repo). What is a phy_repo? a physical one.
same package, different method and line: hook_1($repo) in line 354.
Method hook_1($repo)
It's quite literally about fixing all the hooks.
Recreates dirs for common and admin hooks.
Rewrites update_hook (common) and post_update_hook (admin).
Sets 755 permissions for both common and admin hooks.
Then using ln_sf it symlinks the folders for common/admin hooks.
ln_sf is in common module, in line 162

Debugging crash during app exit (WPF)

I'm trying to figure out why an WPF-app won't exit imediately on closing it. Using Process Explorer I hade found out that WerFault.exe is started while exiting which seem to indicate that something crashes during the teardown, perhaps some destructor or dispose that fails. This started happening when I recently switched to VS2015. I am running Windows 8.
My question is: How can I find out what the real problem is? Any way of finding a crash log for WerFault.exe? I have hundreds of destructors and dispose-methods so it's a bit hard to put breakpoints in all of them. Any other way of capturing these kinds of errors in VS?
The exit code is -1073740791 which "indicate a bug in the executed software that causes stack overflow, leading to abnormal termination of the software". But where?
Some more info from the event log:
Faulting module name: ucrtbase.DLL, version: 10.0.10240.16390, time stamp: 0x55a5b718
Exception code: 0xc0000409
Fault offset: 0x0000000000065a4e
You could try enabling user mode dumps:
Create the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps
Within LocalDumps, create a key that is the name of your executable
Within the key you just created, set the values of DumpFolder, DumpCount, DumpType, and CustomDumpFlags as needed (you should definitely set DumpType to 2 for full dumps, otherwise I don't think that enough information will be captured to debug a managed dump).
Once you have done this, whenever your executable crashes a dump file will be created in the folder specified by DumpFolder (or %LOCALAPPDATA%\CrashDumps by default).

Rsyslog's imfile plugin not working on CentOS 7?

I am trying to get Rsyslog's imfile plugin working without
any real success.
Here is useful OS version information:
# cat /etc/centos-release
CentOS Linux release 7.1.1503 (Core)
And here is Rsyslog version information:
# rsyslogd -v
rsyslogd 7.4.7, compiled with:
FEATURE_REGEXP: Yes
FEATURE_LARGEFILE: No
GSSAPI Kerberos 5 support: Yes
FEATURE_DEBUG (debug build, slow code): No
32bit Atomic operations supported: Yes
64bit Atomic operations supported: Yes
Runtime Instrumentation (slow code): No
uuid support: Yes
See http://www.rsyslog.com for more information.
I tried both legacy and RainerScript format of the configuration.
None of them works for me, sadly. I must be doing something completely wrong
but I simply can not decide on what it could be.
Here is my actual testing configuration (in RainerScript, the
former legacy version I tested was exactly the same in it's meaning):
# cat /etc/rsyslog.conf
global(
workDirectory = "/tmp"
)
module(
load = "imuxsock"
)
module(
load = "imjournal"
stateFile = "journal.state"
)
module(
load = "imfile"
pollingInterval = "10"
)
ruleset(name = "test-ruleset") {
if $syslogtag contains "test-syslogtag" then {
action(
type = "omfile"
file = "/tmp/test-file.log"
)
stop
}
}
input(
type = "imfile"
tag = "test-syslogtag"
stateFile = "test-input.state"
facility = "daemon"
severity = "debug"
file = "/tmp/test-input.in"
ruleset = "test-ruleset"
)
if prifilt("*.*") then {
action(
type = "omfile"
file = "/tmp/rsyslog-testing.log"
)
}
No warning nor error are produced by the Rsyslog with the above
configuration but also nothing from the /tmp/test-input.in file
is copied to the /tmp/test-file.log.
(I also double-checked the /var/log/audit/audit.log, of course, and ...
nothing suspicious is there. Being desperate on what's going on, I also
tried to setenforce 0 to switch SELinux off completely and to restart
the Rsyslog afterwards. It did not helped so the root cause of the problem
may not be SELinux-related issue.)
Also, the test-input.state file is correctly created in the global
workDirectory path (/tmp in this testing case). I also tried
standard paths (logs in /var/log, state file in /var/lib/rsyslog)
and it does not work either although all related files were created properly.
What's weird: I can not see any change in the state file if I populate the
input log file with some testing data even after Rsyslog restart using
# systemctl restart rsyslog (it should update the state file by default).
Just to point out: the imjournal and imuxsock plugins work and populate the fallback log file /tmp/rsyslog-testing.log correctly. Also
manually running Rsyslog on foreground with -D and/or -d options
did not helped me much to clarify why the imfile plugin does not work
for me in this particular configuration.
So, could you please
check my RainerScript syntax whether there is no obvious fault (I guess there is no such),
show me some working imfile plugin configuration on EL7?
Thank you very much.
--
mjf
With a few minor changes it finaly started to work properly. I think the
main root cause of the problem in my case must have been my testing it in
the /tmp directory where Rsyslog does not seem to work properly for some
reason on CentOS 7.
(May it be the /tmp is populated by the File System Namespace even
despite the fact that Systemd option PrivateTmp is not set to true in
the Rsyslog unit file and this option should be set to false by
default according to the Systemd manual page? This is higly unprobable, but
I haven't managed myself to dig more further into it yet. If I find it out,
I will update this answer.)
The other minor cause might have been incorrect filter written in
RainerScript (my real testing instance contained a horrible typo I
simply over-looked). So here is the resulting testing configuration that
works like charm for me.
# cat /etc/rsyslog.conf
global(
workDirectory = "/var/lib/rsyslog"
)
module(
load = "imuxsock"
)
module(
load = "imjournal"
stateFile = "journal.state"
)
module(
load = "imfile"
pollingInterval = "10"
)
ruleset(name = "test-ruleset") {
if $programname == "test-syslogtag" then {
action(
type = "omfile"
file = "/var/log/test-file.log"
)
stop
}
}
input(
type = "imfile"
tag = "test-syslogtag:"
stateFile = "test-input.state"
facility = "daemon"
severity = "debug"
file = "/var/log/test-input.in"
ruleset = "test-ruleset"
)
if prifilt("*.*") then {
action(
type = "omfile"
file = "/var/log/rsyslog-testing.log"
)
}
A little hint for those not knowing it - the $syslogtag and the
$programname seem to be close relatives: $syslogtag := $programname ":".
You can easily find out all the $ prefixed variables you can match against
by using RSYSLOG_DebugFormat output template which is already compiled in.
I hope it helps.
--
mjf

Undocumented Managed VM task queue RPCFailedError

I'm running into a very peculiar and undocumented issue with a GAE Managed VM and Task Queues. I understand that the Managed VM service is in beta, so this question may not be relevant forever, but it's definitely causing me lots of headache now.
The main symptom of the issue is that, in certain (not completely known to me) circumstances, I'm seeing the following error/traceback:
File "/home/vmagent/my_app/some_file.py", line 265, in some_ndb_tasklet
res = yield some_task.add_async('some-task-queue-name')
File "/home/vmagent/python_vm_runtime/google/appengine/ext/ndb/tasklets.py", line 472, in _on_rpc_completion
result = rpc.get_result()
File "/home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/home/vmagent/python_vm_runtime/google/appengine/api/taskqueue/taskqueue.py", line 1948, in ResultHook
rpc.check_success()
File "/home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success
self.__rpc.CheckSuccess()
File "/home/vmagent/python_vm_runtime/google/appengine/ext/vmruntime/vmstub.py", line 312, in _WaitImpl
raise self._ErrorException(*_DEFAULT_EXCEPTION)
RPCFailedError: The remote RPC to the application server failed for call taskqueue.BulkAdd().
I've gone through my local App Engine SDK to trace this through, and I can get up to the last line of the trace, but google/appengine/ext/vmruntime/ doesn't exist on my machine at all, so I have no idea what's happening in vmstub.py. From looking at the local code, some_task.add_async('the-queue') is spinning up an RPC and waiting for it to finish, but this error is not what the except apiproxy_errors.ApplicationError, e: at line 1949 of taskqueue.py is expecting...
The code that's generating the error looks something like this:
#ndb.tasklet
def kickoff_tasks(batch_of_payloads):
for task_payload in batch_of_payloads:
# task_payload is a dict
task = taskqueue.Task(
url='/the/handler/url',
params=payload)
res = yield task.add_async('some-valid-task-queue-name')
Other things worth noting:
this code itself is running in a task handler kicked off by another task.
I first saw this error before implementing this sort of batching, and assumed the issue was because I had added too many tasks from within a task handler.
In some cases, I can run this successfully with a batch size of 100, but in others, it fails consistently (depending on the data in the payloads) at 100, and sometimes succeeds at batch sizes of 50.
The task payloads themselves include batches of items, and are tuned to be just small enough to fit in a task. App Engine advertises a maximum task size of 100KB, so I'm keeping the payloads to under 90,000 bytes right now. Lowering the size even more doesn't seem to help any.
I've also tried implementing an exponential backoff to retry the kickoff_tasks method when this error appears, but it seems that once the error is raised, I can't add any other tasks at all from within the same handler (i.e. I can't kickoff a "continue from where you left off" task, I just have to let this one fail and restart itself)
So, my question is, what is actually causing this error? How can I avoid it, or fix this so that I'm handling it correctly?
This is a known issue that is being worked on. There are actually two issues - the RPC failure itself and the lack of handling of the RPCFailedError exception by the SDK.
There is some public discussion of the issue here.
If you're using App Engine Flexible and the python-compat-multicore image, a new bug popped up related to App Engine using a newer version of the requests library that broke the communication between App Engine Flexible and the datastore. You can fix this error by monkey patching the library in your appengine_config.py file.
Add the following code to appengine_config.py:
try:
import appengine.ext.vmruntime.vmstub as vmstub
except ImportError:
pass
else:
if isinstance(vmstub.DEFAULT_TIMEOUT, (int, long)):
# Newer requests libraries do not accept integers as header values.
# Be sure to convert the header value before sending.
# See Support Case ID 11235929.
vmstub.DEFAULT_TIMEOUT = bytes(vmstub.DEFAULT_TIMEOUT)
Note that if you do not have an appengine_config.py file, you can just create it in your base project directory (wherever you put your app.yaml file). This file gets run during App Engine startup..

Resources