What are the sign extension rules for calling Windows API functions (stdcall)? This is needed to call WInAPI from Go, which is strict about int types - c

Oops, there was one thing I forgot when I made this answer, and it's something that I'm both not quite sure on myself and that I can't seem to find information for on MSDN and Google and the Stack Overflow search.
There are a number of places in the Windows API where you use a negative number, or a number too large to fit in a signed integer; for instance, CW_USEDEFAULT, INVALID_HANDLE_VALUE, GWLP_USERDATA, and so on. In the world of C, everything is all fine and dandy: the language's integer promotion rules come to the rescue.
But in Go, I have to pass all my arguments to functions as uintptr (which is equivalent to C's uintptr_t). The return value from the function is also returned this way, and then I will need to compare. Go doesn't allow integer promotion, and it doesn't allow you to convert a signed constant expression into an unsigned one at compile-time.
Right now, I have a bit of a jerry-rig set up for handling these constants in my UI library. (Here's an example of what this solution looks like in action.) However, I'm not quite satisfied with this solution; it feels to me like it's assuming things about the ABI, and I want to be absolutely sure of what I'm doing.
So my question is: how are signed values handled when passing them to Windows API functions and how are they handled when returning?
All my constants are autogenerated (example output). The autogenerator uses a C ffi, which I'd rather not use for the main project since I can call the DLLs directly (this also makes cross-compilation easier at least for the rest of the year). If I could somehow leverage that, for instance by making everything into a C-side variable of the form
uintptr_t x_CONST_NAME = (uintptr_t) (CONST_NAME);
that would be helpful. But I can't do that without this answer.
Thanks!
Update
Someone on IRC put it differently (reformatted to avoid horizontal scrolling):
[19:13] <FraGag> basically, you're asking whether an int with a value of -1
will be returned as 0x00000000FFFFFFFF or as 0xFFFFFFFFFFFFFFFF
if an int is 4 bytes and an uintptr is 8 bytes
Basically this, but specifically for Windows API interop, for parameters passed in, and regardless of uintptr size.

#twotwotwo's comments to my question pointed me in the right direction. If Stack Overflow allowed marking comments as answers and having multiple answers marked, I'd do that.
tl;dr version: what I have now is correct after all.
I wrote a program (below) that simply dumped all the constants from package syscall and looked for constants that were negative, but not == -1 (as that would just be ^0). The standard file handles (STD_ERROR_HANDLE, STD_INPUT_HANDLE, and STD_OUTPUT_HANDLE) are (-12, -10, and -11, respectively). The code in package syscall passes these constants as the sole argument of getStdHandle(h int), which produces the required file handle for package os. getStdHandle() passes this int to an autogenerated function GetStdHandle(stdhandle int) that wraps a call to the GetStdHandle() system call. GetStdHandle() takes the int and merely converts it to uintptr for passing into syscall.Syscall(). Though no explanation is given in the autogenerator's source (mksyscall_windows.go), if this didn't work, neither would fmt.Println() =P
All of the above is identical on both windows/386 and windows/amd64; the only thing in a processor-specific file is GetStdHandle(), but the relevant code is identical.
My negConst() function is already doing the same thing, just more directly. As such, I can safely assume that it is correct.
Thanks!
// 4 june 2014
// based on code from 24 may 2014
package main
import (
"fmt"
"os"
"strings"
"go/token"
"go/ast"
"go/parser"
"code.google.com/p/go.tools/go/types"
_ "code.google.com/p/go.tools/go/gcimporter"
)
var arch string
func getPackage(path string) (typespkg *types.Package, pkginfo types.Info) {
var pkg *ast.Package
fileset := token.NewFileSet() // parser.ParseDir() actually writes to this; not sure why it doesn't return one instead
filter := func(i os.FileInfo) bool {
if strings.Contains(i.Name(), "_windows") &&
strings.Contains(i.Name(), "_" + arch) &&
strings.HasSuffix(i.Name(), ".go") {
return true
}
if i.Name() == "race.go" || // skip these
i.Name() == "flock.go" {
return false
}
return strings.HasSuffix(i.Name(), "_windows.go") ||
(!strings.Contains(i.Name(), "_"))
}
pkgs, err := parser.ParseDir(fileset, path, filter, parser.AllErrors)
if err != nil {
panic(err)
}
for k, _ := range pkgs { // get the sole key
if pkgs[k].Name == "syscall" {
pkg = pkgs[k]
break
}
}
if pkg == nil {
panic("package syscall not found")
}
// we can't pass pkg.Files directly to types.Check() because the former is a map and the latter is a slice
ff := make([]*ast.File, 0, len(pkg.Files))
for _, v := range pkg.Files {
ff = append(ff, v)
}
// if we don't make() each map, package types won't fill the structure
pkginfo.Defs = make(map[*ast.Ident]types.Object)
pkginfo.Scopes = make(map[ast.Node]*types.Scope)
typespkg, err = new(types.Config).Check(path, fileset, ff, &pkginfo)
if err != nil {
panic(err)
}
return typespkg, pkginfo
}
func main() {
pkgpath := "/home/pietro/go/src/pkg/syscall"
arch = os.Args[1]
pkg, _ := getPackage(pkgpath)
scope := pkg.Scope()
for _, name := range scope.Names() {
obj := scope.Lookup(name)
if obj == nil {
panic(fmt.Errorf("nil object %q from scope %v", name, scope))
}
if !obj.Exported() { // exported names only
continue
}
if _, ok := obj.(*types.Const); ok {
fmt.Printf("egrep -rh '#define[ ]+%s' ~/winshare/Include/ 2>/dev/null\n", obj.Name())
}
// otherwise skip
}
}

Related

String.withCString when the String is nil

The problem that'll be described relates to my previous question:
string.withCString and UnsafeMutablePointer(mutating: cstring) wrapped into a function which was my first approach to handle nil Strings (by putting withCString into a function)
and to a question which Mecki asked:
Why can't I pass an optional Swift String to C function that allows NULL pointers?
Imagine there is a c-function like:
unsigned long randomSign(char *pin, char *tag_signature, char *tag_data, char *xyz);
I know that the function works correctly if I wrap 4 string.withCString closures around the corresponding swift function:
// pin, tag_signature, tag_data and xyz are optional Strings so they may be nil which is a problem for my result.
// corresponding swift function:
// randomSign(pin: UnsafeMutablePointer<Int8>!, tag_signature: UnsafeMutablePointer<Int8>!, tag_data: UnsafeMutablePointer<Int8>!, xyz: UnsafeMutablePointer<Int8>!)
let result = pin.withCString { s1 in return
tag_signature.withCString {s2 in return
tag_data.withCString {s3 in return
xyz.withCString { s4 in return
randomSign(UnsafeMutablePointer(mutating: s1), UnsafeMutablePointer(mutating: s2), UnsafeMutablePointer(mutating: s3), UnsafeMutablePointer(mutating: s4))
}}}}
And so Martin R replied to an easier example, that it is not needed to wrap the closures around randomSign(arguments) and UnsafeMutablePointer(mutating: ...) because it can also take the strings and converts it.
But when I drop the closures and use it as Martin R described, it worked at the first launch on the simulator directly after starting the mac, but on consecutive calls of the randomSign-Function the return would tell me that for example the tag_signature or the pin would be invalid (but it actually is valid and I don't know why?!).
This leads me to the problem that I need the withCString closures (at the moment) but I have to handle nil-Strings, which would result the app to crash when it shall return the result because it couldn't evaluate the randomSign-Function.
So I tried to fit the approach below (also suggested by #Martin R) to Swift 3, but I did not workout to adapt it.
//Swift-2 written by Martin R
protocol CStringConvertible {
func withCString<Result>(#noescape f: UnsafePointer<Int8> throws -> Result) rethrows -> Result
}
extension String: CStringConvertible { }
extension Optional where Wrapped: CStringConvertible {
func withOptionalCString<Result>(#noescape f: UnsafePointer<Int8> -> Result) -> Result {
if let string = self {
return string.withCString(f)
} else {
return f(nil)
}
}
}
//Swift 3: ???
If anyone can tell me, why my function only works out when I use withCString but not when I dismiss it, I would be very grateful
and also if anyone knows how to solve the issue, i.e. correctly translating the swift-2 code to working swift-3 code.
The problem with
let result = randomSign(UnsafeMutablePointer(mutating: pin),
UnsafeMutablePointer(mutating: tag_signature),
UnsafeMutablePointer(mutating: tag_data),
UnsafeMutablePointer(mutating: xyz))
is that the temporary UTF-8 representation created from the Swift
strings is valid only during each call of UnsafeMutablePointer(),
but not necessarily still valid during the call of randomSign().
(So my final suggestion in https://stackoverflow.com/a/44027397/1187415
was actually not correct, I have updated that part).
A possible Swift 3 version of the wrapper in https://stackoverflow.com/a/39363769/1187415 is
extension Optional where Wrapped == String {
func withOptionalCString<Result>(_ f: (UnsafeMutablePointer<Int8>?) -> Result) -> Result {
if let string = self {
return string.withCString { f(UnsafeMutablePointer(mutating: $0)) }
} else {
return f(nil)
}
}
}
This handles both the optionality and casts the C string pointer
to a mutable pointer (as required by randomSign()). This can be
called as
let result = pin.withOptionalCString { s1 in
tag_signature.withOptionalCString { s2 in
tag_data.withOptionalCString { s3 in
xyz.withOptionalCString { s4 in
randomSign(s1, s2, s3, s4)
}
}
}
}
Remark: In theory, the problem can be avoided if the signature of randomSign() is changed to take const char * parameters:
unsigned long randomSign(const char *pin, const char *tag_signature, const char *tag_data, const char *xyz);
which one could then simply call as
let result = randomSign(pin, tag_signature, tag_data, xyz)
with optional or non-optional Swift strings.
However, this does currently not work, as reported in
SR-2814 Swift does not correctly pass in multiple optional strings to C function.

Golang implementing database funcs using interfaces

Sorry for a bit dummy questions, but I'm kinda stuck.
So, I'm implementing wrapper above database driver for my application, and I need to keep it as portable as possible.
I came to decision that interfaces are perfect match for this task. So, I have my database struct with some variables and app-specific methods, and two interface functions:
query(request string) error
flush() int? string?? struct?? slice????, error
Now you probably got the main question. How do I do return data that type "flush()" doesn't know? Can I return it by interface, and if I can, how to work with that?
Second question is pretty basic, but still it isn't clear to me.
So, I have this database struct with two methods designed to be implemented by package user to use db driver he want.
Ho do I write it and how future implementation will look (there is an example on tour of go, but it's about interface for different structs with similar methods)
Hope you'll help me find an understanding :)
Yes, flush can just have the signiture;
flush() interface{}, error
How do you implement? Something like this with reasonable method bodies should do it for you;
type MyDbDriver struct {
//fields
}
func (d *MyDbDriver) query(request string) error {
return nil
}
func (d *MyDbDriver) flush() interface{}, error {
return nil, nil
}
In Go all interface implementations are implicit, meaning, if your type has methods that match the interfaces signature, then you've implemented it. No need for something like public class MyType: IMyInterface, IThisIsntCSharp. Note that in the example above *MyDbDriver has implemented your interface but MyDbDriver has not.
Edit: below a some pseudo calling code;
e := driver.query("select * from thatTable where ThatProperty = 'ThatValue'")
if e != nil {
return nil, e
}
i, err := driver.flush()
if err != nil {
return nil, err
}
MyConcreteInstance := i.(MyConcreteType)
// note that this will panic if the type of i is not MyConcreteType
// that can be avoided with the familiar object, err calling syntax
MyConcreteIntance, ok := i.(MyConcreteType)
if !ok {
// the type of i was not MyConcreteType :\
}

Why do I get different results when using libiconv over iconv binary?

Here is the sample string that I am using encoded in UCS-2:
abvgdđežzijklmnjoprstćuvhcčdžš1234567890*+;'
When converting UCS-2 to iso ISO-8859-1//TRANSLIT with iconv binary from file to file I get:
abvgd?ezzijklmnjoprstcuvhccdzs1234567890*+;'
Now I want to use libiconv in go project. I am using this library github.com/qiniu/iconv as bindings for libiconv. But when using bindings I get:
abvgd?e?zijklmnjoprst?uvhc?d??1234567890*+;'
It's like different transliteration rules apply when using library inside go.
I examined go bindings library and everything seems in order; only bytes are passed around so no "loss of information" could happen there.
Is there anything else that I should be aware of when using libiconv? Is there some environment context that could trigger different transliteration behaviour?
EDIT (additional explanation about invocation):
I have two files "ucs-2.txt" and "latin1.txt". ucs-2.txt file contains UCS-2 encoded string and latin1.txt contains string got by running:
iconv -f UCS2 -t ISO-8859-1//TRANSLIT --verbose data/encoding/ucs-2.txt > data/encoding/latin1.txt
In go I use these lines to pull content from these files:
var err error
ucs2, err = ioutil.ReadFile("data/encoding/ucs-2.txt")
if err != nil {
log.Fatal(err)
}
latin1, err = ioutil.ReadFile("data/encoding/latin1.txt")
if err != nil {
log.Fatal(err)
}
This function is doing conversion:
func convertEnc(content []byte) ([]byte, error) {
cd, err := iconv.Open("ISO-8859-1//TRANSLIT", "UCS2")
if err != nil {
return nil, err
}
defer cd.Close()
var outbuf [255]byte
res, _, err := cd.Conv(content, outbuf[:])
log.Printf("result: %+q", res)
return res, err
}
And I am using DeepEqual for testing:
reflect.DeepEqual(res, latin1)
The first output includes transliteration, i.e. certain characters (e.g. ž) are transliterated into their not-quite-right "plain" counterpart (z) in order to be representable in an encoding that does not support the original character (here, ž in Latin-1).
The second output did not transliterate anything, it dropped any characters not representable in the target encoding (ž, ć, ... in Latin-1).
Thus, I suspect you can the binary with different options than the library. Not familiar with libiconv, it seems that the //TRANSLIT part was omitted or is not supported by the function you used...?
Transliteration is locale dependent. May be your libiconv is lacking/has wrong locale. Or the locale you are using there has no transliteration configured.
Please check this bug report as it has a few examples and a discussion on this topic.

_file_ or _line_ similar in golang

is there any function in go that is similar like "_file_" or "_line_" in go, to know who is calling a specific function during run time? In C we have the "_file_" line that can be called as macros. How to do this in go?
If you're using the log package, you can instruct the logger to prefix the entries with various information. You'll likely be most interested in the Lshortfile constant, which will result in prefixes along the lines of d.go:23. Alternatively, there is Llongfile which prints the file's full path (such as /a/b/c/d.go:23).
If you don't want to use the log package, you could also use runtime.Caller(), which is what the log package uses internally. It's not as straight forward as the C macros, but you could hide it behind a function (and specify the correct call depth). You can see how the log package is implemented for an example (line 140).
(1) Write a brief function that calls runtime.Caller()
(2) Call that function everywhere you want to access the source code file and line number at run time.
Example:
import "runtime"
func file_line() string {
_, fileName, fileLine, ok := runtime.Caller(1)
var s string
if ok {
s = fmt.Sprintf("%s:%d", fileName, fileLine)
} else {
s = ""
}
return s
}
Note: pass 1 to Caller() so that it returns the number of the line where file_line() is called, instead of where runtime.Caller() is called.
fmt.Println(file_line()) // Prints this file and line number.
See the runtime and runtime.debug packages and in particular the Stack, PrintStack or Callerfunctions.
Stack formats a stack trace of the calling goroutine into buf and returns the number of bytes written to buf. If all is true, Stack formats stack traces of all other goroutines into buf after the trace for the current goroutine.
If you are compiling with debug information, then this will should contain the line number in source
//import ( "log" "strings" "bytes" "path/filepath" )
func SRC_FILE() string {
var buf bytes.Buffer
log.New(&buf, "", log.Lshortfile).Output(2, "")//2=>CALLER
__FILE_LINE__ := strings.TrimSpace(buf.String())
__FILE__ := strings.TrimRight(__FILE_LINE__, ":1234567890")
return __FILE__
}
func SRC_DIR() string {
var buf bytes.Buffer
log.New(&buf, "", log.Llongfile).Output(2, "")//2=>CALLER
__FILE_LINE__ := strings.TrimSpace(buf.String())
__FILE__ := strings.TrimRight(__FILE_LINE__, ":1234567890")
__DIR__ := filepath.Dir(__FILE__)
return __DIR__
}
func main() {
fmt.Println(SRC_FILE())//analogous to __FILE__
fmt.Println(SRC_DIR()) //analogous to __DIR__
}
https://go.dev/play/p/3nvgknX96RO
When you run it on go.dev/play your code is run as prog.go

Wrapping FUSE from Go

I'm playing around with wrapping FUSE with Go. However I've come stuck with how to deal with struct fuse_operations. I can't seem to expose the operations struct by declaring type Operations C.struct_fuse_operations as the members are lower case, and my pure-Go sources would have to use C-hackery to set the members anyway. My first error in this case is "can't set getattr" in what looks to be the Go equivalent of a default copy constructor. My next attempt is to expose an interface that expects GetAttr, ReadLink etc, and then generate C.struct_fuse_operations and bind the function pointers to closures that call the given interface.
This is what I've got (explanation continues after code):
package fuse
// #include <fuse.h>
// #include <stdlib.h>
import "C"
import (
//"fmt"
"os"
"unsafe"
)
type Operations interface {
GetAttr(string, *os.FileInfo) int
}
func Main(args []string, ops Operations) int {
argv := make([]*C.char, len(args) + 1)
for i, s := range args {
p := C.CString(s)
defer C.free(unsafe.Pointer(p))
argv[i] = p
}
cop := new(C.struct_fuse_operations)
cop.getattr = func(*C.char, *C.struct_stat) int {}
argc := C.int(len(args))
return int(C.fuse_main_real(argc, &argv[0], cop, C.size_t(unsafe.Sizeof(cop)), nil))
}
package main
import (
"fmt"
"fuse"
"os"
)
type CpfsOps struct {
a int
}
func (me *CpfsOps) GetAttr(string, *os.FileInfo) int {
return -1;
}
func main() {
fmt.Println(os.Args)
ops := &CpfsOps{}
fmt.Println("fuse main returned", fuse.Main(os.Args, ops))
}
This gives the following error:
fuse.go:21[fuse.cgo1.go:23]: cannot use func literal (type func(*_Ctype_char, *_Ctype_struct_stat) int) as type *[0]uint8 in assignment
I'm not sure what to pass to these members of C.struct_fuse_operations, and I've seen mention in a few places it's not possible to call from C back into Go code.
If it is possible, what should I do? How can I provide the "default" values for interface functions that acts as though the corresponding C.struct_fuse_operations member is set to NULL?
http://cheesesun.blogspot.com/2010/04/callbacks-in-cgo.html describes a general - if somewhat convoluted - technique. http://groups.google.com/group/golang-nuts/browse_thread/thread/abd0e30dafdbf297?tvc=2&pli=1 describes the problem.
I suspect that if you want to handle the function pointers as anything other than uintptrs, you'll have to hack on cgo.

Resources