How to make scanning DB rows in Go DRY? - database

Question

I have a table in the database containing user account information. I have a struct called User defined.
type User struct {
Id uint
Username string
Password string
FirstName string
LastName string
Address1 string
Address2 string
.... a bunch more fields ...
}
For fetching individual user accounts, I have a method defined
func (user *User) GetById(db *sql.DB, id uint) error {
query := `SELECT
...a whole bunch of SQL ...
WHERE id = $1
... more SQL ...
LIMIT 1`
row := db.QueryRow(query, id)
err := row.Scan(
&user.Id,
&user.UserName,
&user.Password,
&user.FirstName,
&user.LastName,
... some 20 more lines of fields read into the struct ...
)
if err != nil {
return err
}
return nil
}
And there are several places in the system where I need to fetch user information as part of a larger query. That is, I am fetching some other type of object, but also a user account related to it.
That means, I have to repeat the whole rows.Scan(&user.Username, &user...) thing over and over again and it takes a whole page and it is error prone and if I ever change the user table structure I would have to change the code in a whole bunch of places. How can I make this more DRY?
Edit: I am not sure why this was marked as a duplicate, but since this edit is required, I will try to explain one more time. I am not asking how to scan a row into a struct. I already know how to do that, as the code above clearly shows. I am asking how to structure the struct scanning code in such a way that I do not have to repeat the same page of scanning code every time I am scanning the same type of struct.
Edit: also, yes, I am aware of sqlstruct and sqlx and similar libraries. I am deliberately avoiding these, because they depend on reflect package with well documented performance issues. And I intend to potentially scan millions of rows using these techniques (not millions of users, but this question extends to other record types).
Edit: so, yes, I know I should write a function. I am not sure what this function should take as arguments and what results it should return. Lets say that the other query I want to accommodate looks like this
SELECT
s.id,
s.name,
... more site fields ...
u.id,
u.username,
... more user fields ...
FROM site AS s
JOIN user AS u ON (u.id = s.user_id)
JOIN some_other_table AS st1 ON (site.id = st1.site_id)
... more SQL ...
And I have a site struct method that embeds a user struct. I don't want to repeat the user scanning code here. I want to call a function that will scan the user portion of the raw into a user struct the same way it does in the user method above.

Answer 1

To eliminate the repetition of the required steps to scan the *sql.Rows structure you could introduce two interfaces. One that describes the already implemented behaviour of *sql.Rows and *sql.Row.
// This interface is already implemented by *sql.Rows and *sql.Row.
type Row interface {
Scan(...interface{}) error
}
And another one that abstracts away the actual scanning step of the row(s).
// have your entity types implement this one
type RowScanner interface {
ScanRow(Row) error
}
An example implementation of the RowScanner interface could look like this:
type User struct {
Id uint
Username string
// ...
}
// Implements RowScanner
func (u *User) ScanRow(r Row) error {
return r.Scan(
&u.Id,
&u.Username,
// ...
)
}
type UserList struct {
Items []*User
}
// Implements RowScanner
func (list *UserList) ScanRow(r Row) error {
u := new(User)
if err := u.ScanRow(r); err != nil {
return err
}
list.Items = append(list.Items, u)
return nil
}
With these interfaces you can now dry your rows-scanning code for all of your types that implement the RowScanner interface by using these two functions.
func queryRows(query string, rs RowScanner, params ...interface{}) error {
rows, err := db.Query(query, params...)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
if err := rs.ScanRow(rows); err != nil {
return err
}
}
return rows.Err()
}
func queryRow(query string, rs RowScanner, params ...interface{}) error {
return rs.ScanRow(db.QueryRow(query, params...))
}
// example
ulist := new(UserList)
if err := queryRows(queryString, ulist, arg1, arg2); err != nil {
panic(err)
}
// or
u := new(User)
if err := queryRow(queryString, u, arg1, arg2); err != nil {
panic(err)
}
If you have composite types that you want to scan but you want to avoid having to repeat the enumeration of its elements' fields, then you could introduce a method that returns a type's fields and reuse that method where you need it. For example:
func (u *User) ScannableFields() []interface{} {
return []interface{}{
&u.Id,
&u.Username,
// ...
}
}
func (u *User) ScanRow(r Row) error {
return r.Scan(u.ScannableFields()...)
}
// your other entity type
type Site struct {
Id uint
Name string
// ...
}
func (s *Site) ScannableFields() []interface{} {
return []interface{}{
&p.Id,
&p.Name,
// ...
}
}
// Implements RowScanner
func (s *Site) ScanRow(r Row) error {
return r.Scan(s.ScannableFields()...)
}
// your composite
type UserWithSite struct {
User *User
Site *Site
}
// Implements RowScanner
func (u *UserWithSite) ScanRow(r Row) error {
u.User = new(User)
u.Site = new(Site)
fields := append(u.User.ScannableFields(), u.Site.ScannableFields()...)
return r.Scan(fields...)
}
// retrieve from db
u := new(UserWithSite)
if err := queryRow(queryString, u, arg1, arg2); err != nil {
panic(err)
}