I need to get new lines of syslog to my c(or golang) program when it written.
The program run as linux daemon, and it will be always on memory.
Here, the picture explains full code flow that I needed.
Run Flows
Please check and guide me how.
regards,
You can use nxadm/tail which mimics UNIX tail. If you need to have a finer grain of control, you can use inotify feature with fsnotify.
Tail:
package main
import (
"fmt"
"github.com/nxadm/tail"
)
func main() {
t, _ := tail.TailFile("/var/log", tail.Config{Follow: true})
for line := range t.Lines {
fmt.Println(line.Text)
}
}
fsnotify:
package main
import (
"log"
"github.com/fsnotify/fsnotify"
)
func main() {
watcher, err := fsnotify.NewWatcher()
if err != nil {
log.Fatal(err)
}
defer watcher.Close()
go func() {
for {
select {
case event, ok := <-watcher.Events:
if !ok {
return
}
log.Println("event:", event)
case err, ok := <-watcher.Errors:
if !ok {
return
}
log.Println("error:", err)
}
}
}()
// Add a path.
err = watcher.Add("/var/log")
if err != nil {
log.Fatal(err)
}
// Block main goroutine forever.
<-make(chan struct{})
}
Related
I have a function that is designed to insert a large number of elements into an MSSQL database using a table-valued parameter and a procedure.
func (requester *Requester) doQuery(ctx context.Context, dtos interface{}) error {
conn, err := requester.conn.Conn(ctx)
if err != nil {
return err
}
defer func() {
if clErr := conn.Close(); clErr != nil {
err = clErr
}
}()
tx, err := conn.BeginTx(ctx, &sql.TxOptions{Isolation: sql.LevelRepeatableRead, ReadOnly: false})
if err != nil {
return err
}
defer func() {
if p := recover(); p != nil {
tx.Rollback()
panic(p)
} else if err != nil {
tx.Rollback()
} else {
tx.Commit()
}()
param := sql.Named("TVP", mssql.TVP{
TypeName: "MyTypeName",
Value: dtos,
})
return tx.ExecContext(ctx, "EXEC [dbo].[usp_InsertConsumption] #TVP", param)
}
The test I wrote for this function is included below (note that it depends on ginkgo and gomega):
Describe("SQL Tests", func() {
It("AddConsumption - No failures - Added", func() {
db, mock, _ := sqlmock.New()
requester := Requester{conn: db}
defer db.Close()
mock.ExpectBegin()
mock.ExpectExec(regexp.QuoteMeta("EXEC [dbo].[usp_InsertConsumption] #TVP")).
WithArgs("").WillReturnResult(sqlmock.NewResult(1, 1))
mock.ExpectExec(regexp.QuoteMeta("EXEC [dbo].[usp_InsertTags] #TVP")).
WithArgs("").WillReturnResult(sqlmock.NewResult(1, 1))
mock.ExpectCommit()
err := requester.doQuery(context.TODO(), generateData())
Expect(err).ShouldNot(HaveOccurred())
Expect(mock.ExpectationsWereMet()).ShouldNot(HaveOccurred())
})
})
Now, this code was written for a MySQL context and since I've ported the code over to MSSQL, I've been getting a peculiar error:
sql: converting argument with name \"TVP\" type: unsupported type mssql.TVP, a struct
It appears that sqlmock is attempting to call ConvertValue on the TVP object, which is invalid. So, how do I make sqlmock handle this value correctly so I can unit test around the query?
What I discovered here is that sqlmock has a function called ValueConverterOption, which, when provided with a custom driver.ValueConverter interface. This will be called in place of the standard function for every invocation of ConvertValue. If you want to test around the ExecContext function when it receives a non-standard argument, a TVP in this case, then you can use this function to inject custom conversion logic into sqlmock.
type mockTvpConverter struct {}
func (converter *mockTvpConverter) ConvertValue(raw interface{}) (driver.Value, error) {
// Since this function will take the place of every call of ConvertValue, we will inevitably
// the fake string we return from this function so we need to check whether we've recieved
// that or a TVP. More extensive logic may be required
switch inner := raw.(type) {
case string:
return raw.(string), nil
case mssql.TVP:
// First, verify the type name
Expect(inner.TypeName).Should(Equal("MyTypeName"))
// VERIFICATION LOGIC HERE
// Finally, return a fake value that we can use when verifying the arguments
return "PASSED", nil
}
// We had an invalid type; return an error
return nil, fmt.Errorf("Invalid type")
}
which means, the test then becomes:
Describe("SQL Tests", func() {
It("AddConsumption - No failures - Added", func() {
db, mock, _ := sqlmock.New(sqlmock.ValueConverterOption(&mockTvpConverter{}))
requester := Requester{conn: db}
defer db.Close()
mock.ExpectBegin()
mock.ExpectExec(regexp.QuoteMeta("EXEC [dbo].[usp_InsertConsumption] #TVP")).
WithArgs("PASSED").WillReturnResult(sqlmock.NewResult(1, 1))
mock.ExpectExec(regexp.QuoteMeta("EXEC [dbo].[usp_InsertTags] #TVP")).
WithArgs("PASSED").WillReturnResult(sqlmock.NewResult(1, 1))
mock.ExpectCommit()
err := requester.doQuery(context.TODO(), generateData())
Expect(err).ShouldNot(HaveOccurred())
Expect(mock.ExpectationsWereMet()).ShouldNot(HaveOccurred())
})
})
I had this on my go project using GORM:
import (
(...)
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
)
func main() {
(...)
db, err := gorm.Open("postgres", dsn)
if err != nil {
panic(err)
}
defer db.Close()
}
I was able to open the connection to the DB and do all my transactions. If I change to:
import (
(...)
"github.com/jinzhu/gorm"
"gorm.io/driver/postgres"
)
func main(){
(...)
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic(err)
}
defer db.Close()
}
it will give me an error on go build:
# gorm.io/driver/postgres
$GOPATH\pkg\mod\gorm.io\driver\postgres#v1.0.5\migrator.go:206:65: undefined: gorm.ColumnType
$GOPATH\pkg\mod\gorm.io\driver\postgres#v1.0.5\migrator.go:207:23: undefined: gorm.ColumnType
I got this from the official page:
https://gorm.io/docs/connecting_to_the_database.html
I changed to using the same library... my bad.
Instead of
"github.com/jinzhu/gorm"
use this:
"gorm.io/gorm"
along the same driver
So now it looks like this:
import (
(...)
"gorm.io/gorm"
"gorm.io/driver/postgres"
)
func main(){
(...)
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic(err)
}
defer db.Close()
}
You might need to upgrade to a newer version of gorm.
I got this error when using v1.20.4.
gorm.io/gorm v1.20.7 works for me
Looking at the helloworld example from grpc-go, how can I add a StopGrpcServer() function to stop gRPC server?
I'm trying to make a shared C library file of gRPC server which can be used to start and stop the gRPC server by invoking the functions via ffi.
In my tests I am able to start the gRPC server with StartGrpcServer() but I'm feeling lost how to implement StopGrpcServer().
package main
import C
import (
"context"
"log"
"net"
"google.golang.org/grpc"
pb "google.golang.org/grpc/examples/helloworld/helloworld"
)
const (
port = ":50051"
)
// server is used to implement helloworld.GreeterServer.
type server struct {
pb.UnimplementedGreeterServer
}
// SayHello implements helloworld.GreeterServer
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
log.Printf("Received: %v", in.GetName())
return &pb.HelloReply{Message: "Hello " + in.GetName()}, nil
}
//export StopGrpcServer
func (s *server) StopGrpcServer() {
s.Stop()
}
//export StartGrpcServer
func StartGrpcServer() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterGreeterServer(s, &server{})
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
func main() {}
Would really appreciate some help on it.
Thanks,
Satinder
Thanks for the hint #Marc. I am able to solve the problem by declaring *grpc.Server variable globally outside StartGrpcServer(), and then using it in StopGrpcServer().
Here's the working code:
package main
import C
import (
"context"
"log"
"net"
"google.golang.org/grpc"
pb "google.golang.org/grpc/examples/helloworld/helloworld"
)
var srv *grpc.Server
const (
port = ":50051"
)
// server is used to implement helloworld.GreeterServer.
type server struct {
pb.UnimplementedGreeterServer
}
// SayHello implements helloworld.GreeterServer
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
log.Printf("Received: %v", in.GetName())
return &pb.HelloReply{Message: "Hello " + in.GetName()}, nil
}
//export StopGrpcServer
func StopGrpcServer() {
srv.Stop()
}
//export StartGrpcServer
func StartGrpcServer() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
srv = grpc.NewServer()
pb.RegisterGreeterServer(srv, &server{})
if err := srv.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
func main() {}
// Compile with command:
// go build -o greeter.so -buildmode=c-shared server.go
TLDR; How to create REST api on model having Foreign key (or db relationship in general) in buffalo framework?
I am absolute beginner in go and I am trying to write an RESTFul service using buffalo framework following following example given on it official website. I am able to create RESTful api on models which have no database relationship. But I am stuck when I faced a situation where model has a foreign key. I am not able to find any documentation or reference on web. I concept on Go is also weak, you can also educate me on those thins.
Models: (ref: https://gobuffalo.io/en/docs/db/relations#example
type Composer struct {
ID uuid.UUID `json:"id" db:"id"`
Name string `json:"name" db:"name"`
Description string `json:"description" db:"description"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
UpdatedAt time.Time `json:"updated_at" db:"updated_at"`
}
type Track struct {
ID uuid.UUID `json:"id" db:"id"`
Title string `json:"title" db:"title"`
Description string `json:"description" db:"description"`
Composer Composer `has_one:"composer" fk_id:"id"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
UpdatedAt time.Time `json:"updated_at" db:"updated_at"`
}
Resources: (ref: https://gobuffalo.io/en/docs/resources)
type TrackResource struct {
buffalo.Resource
}
func (v TrackResource) List(c buffalo.Context) error {
tx, ok := c.Value("tx").(*pop.Connection)
if !ok {
return errors.WithStack(errors.New("no transaction found"))
}
pieces := &models.Tracks{}
q := tx.PaginateFromParams(c.Params())
if err := q.All(pieces); err != nil {
return errors.WithStack(err)
}
c.Set("pagination", q.Paginator)
return c.Render(200, r.JSON(pieces))
}
func (v TrackResource) Show(c buffalo.Context) error {
tx, ok := c.Value("tx").(*pop.Connection)
if !ok {
return errors.WithStack(errors.New("no transaction found"))
}
piece := &models.Track{}
if err := tx.Find(piece, c.Param("track_id")); err != nil {
return c.Render(404, r.JSON(err))
}
return c.Render(200, r.JSON(piece))
}
func (v TrackResource) Create(c buffalo.Context) error {
piece := &models.Track{}
if err := c.Bind(piece); err != nil {
return errors.WithStack(err)
}
tx, ok := c.Value("tx").(*pop.Connection)
if !ok {
return errors.WithStack(errors.New("no transaction found"))
}
verrs, err := piece.Create(tx)
if err != nil {
return errors.WithStack(err)
}
if verrs.HasAny() {
return c.Render(422, r.JSON(verrs))
}
return c.Render(201, r.Auto(c, piece))
}
func (v TrackResource) Update(c buffalo.Context) error {
tx, ok := c.Value("tx").(*pop.Connection)
if !ok {
return errors.WithStack(errors.New("no transaction found"))
}
piece := &models.Track{}
if err := tx.Find(piece, c.Param("track_id")); err != nil {
return c.Error(404, err)
}
if err := c.Bind(piece); err != nil {
return errors.WithStack(err)
}
verrs, err := piece.Update(tx)
if err != nil {
return errors.WithStack(err)
}
if verrs.HasAny() {
return c.Render(422, r.JSON(verrs))
}
return c.Render(200, r.JSON(piece))
}
func (v TrackResource) Destroy(c buffalo.Context) error {
tx, ok := c.Value("tx").(*pop.Connection)
if !ok {
return errors.WithStack(errors.New("no transaction found"))
}
piece := &models.Track{}
if err := tx.Find(piece, c.Param("track_id")); err != nil {
return c.Error(404, err)
}
if err := tx.Destroy(piece); err != nil {
return errors.WithStack(err)
}
return c.Render(200, r.JSON(piece))
}
When I am trying to create a track. I am getting error:
json: cannot unmarshal string into Go struct field Track.Composer of type models.Composer
gitlab.com/****/****/actions.TrackResource.Create
Please help.
You're trying to bind a Track containing a Composer with its ID, but Composer is defined as a struct.
To make it work, you need to implement the Unmarshaler interface and define how to convert this ID into the Composer struct you want.
You should add a ComposerID field to your Track struct. Then you can use Eager or Load, and pop will load the associated Composer object for you. This is shown on the page you've linked to for your Models reference.
Is there a way to determine whether my *File is pointing to a file or a directory?
fileOrDir, err := os.Open(name)
// How do I know whether I have a file or directory?
I want to be able to read stats about the file if it is just a file, and be able to read the files within the directory if it is a directory
fileOrDir.Readdirnames(0) // If dir
os.Stat(name) // If file
For example,
package main
import (
"fmt"
"os"
)
func main() {
name := "FileOrDir"
fi, err := os.Stat(name)
if err != nil {
fmt.Println(err)
return
}
switch mode := fi.Mode(); {
case mode.IsDir():
// do directory stuff
fmt.Println("directory")
case mode.IsRegular():
// do file stuff
fmt.Println("file")
}
}
Note:
The example is for Go 1.1. For Go 1.0, replace case mode.IsRegular(): with case mode&os.ModeType == 0:.
Here is another possibility:
import "os"
func IsDirectory(path string) (bool, error) {
fileInfo, err := os.Stat(path)
if err != nil{
return false, err
}
return fileInfo.IsDir(), err
}
Here is how to do the test in one line:
if info, err := os.Stat(path); err == nil && info.IsDir() {
...
}
fileOrDir, err := os.Open(name)
if err != nil {
....
}
info, err := fileOrDir.Stat()
if err != nil {
....
}
if info.IsDir() {
....
} else {
...
}
Be careful to not open and stat the file by name. This will produce a race condition with potential security implications.
If your open succeeds then your have a valid file handle and you should use the Stat() method on it to obtain the stat. The top answer is risky because they suggest to call os.Stat() first and then presumably os.Open() but someone could change the file in between the two calls.
import "os"
// FileExists reports whether the named file exists as a boolean
func FileExists(name string) bool {
if fi, err := os.Stat(name); err == nil {
if fi.Mode().IsRegular() {
return true
}
}
return false
}
// DirExists reports whether the dir exists as a boolean
func DirExists(name string) bool {
if fi, err := os.Stat(name); err == nil {
if fi.Mode().IsDir() {
return true
}
}
return false
}