Working on an app in Kotlin, using Room. Previously, the table name was "favoritelist" as a string. I moved all API info and database strings to const val. After doing so, I accidentally changed the table name to "favoriteslist". I received reports that the app was crashing after the newest update. How can I get the database table corrected to avoid further crashes? Should I rename it, auto migrate to new version, then release?
You have two potential fixes to cater for. Those who have migrated and those who have newly installed the App.
For the latter the App is probably working fine, but they will be using the newer table name.
For those who have migrated, the crash should have been post migration, after which the comparison against the compiled hash of the schema (what Room expects), is compared with the actual schema that was found.
If this is the case then changes to the database should be rolled back, as if the migration never happened.
Your next decision will be which table name is going to be used. This would be a factor in determining the conditions.
If the crash is as suggested above then you could just use the newer name and have a migration that renames the table from the older to the newer name. But this being a newer version of the App. In which case there would be no Room migration needed for the newer users.
If you want to revert to the original table name, then you would have to rename the table for the new users and the older users could continue but the migration would have to detect or handle the fact that there will be no newer table name for older users.
Obviously the most important factor is that you thoroughly test the scenarios before publishing the App.
Example
The Entity in question being (along with some constants) :-
const val DATABASE_NAME = "the_database.db"
const val DATABASE_VERSION = 3
const val FAVOURITES_TABLE_NAME = "favouritelist" /* "favouriteslist"*/
const val incorrect_table_name = "favouriteslist"
const val FAVOURITES_ID_COLUMN = FAVOURITES_TABLE_NAME + "_id"
const val FAVOURITES_DESCRIPTION_COLUMN = FAVOURITES_TABLE_NAME + "_desc"
#Entity(tableName = FAVOURITES_TABLE_NAME)
data class Favourite(
#PrimaryKey
#ColumnInfo(name = FAVOURITES_ID_COLUMN)
var id: Long?=null,
#ColumnInfo(name = FAVOURITES_DESCRIPTION_COLUMN)
var description: String
)
When first run at version 1 DATABASE_VERSION = 1 rather than DATABASE_VERSION = 3 on one of two devices (no data added just forced and open)
Then when going to Version 2 DATABASE_VERSION = 2
const val FAVOURITES_TABLE_NAME = "favouritelist" /* "favouriteslist"*/
was changed to
const val FAVOURITES_TABLE_NAME = /*"favouritelist"*/ "favouriteslist"
and then run on both devices.
The second device ran OK, reflecting a new install of the App.
The first device failed with the expected but found and the database was rolled back.
So the situation is the old install on the first device is still at version 1 and the new install on the second device is at version 2.
The Fix (to retain the original names)
Version 3 will correct that. So There needs to be two corrective actions:-
for device 1 still on version 1 then effectively the migration from 1 to 3 needs to be skipped.
for device 2 the migration has to rename the tables (and due to the inclusion of the table name in the column names, the columns have to be renamed)
Device 1 aka (1 to 3)
Automigration is perhaps possible but it complained and appeared to want to change to version so it could then change to version 3 (because it progresses through the stored (exported) schemas).
Rather than mess around, knowing that nothing really needs to be done, a manual migration for 1 to 3 was added :-
.addMigrations(Migration(1,3){})
Device 2 aka (2 to 3) first :-
#RenameColumn(incorrect_table_name, incorrect_table_name+"_id", toColumnName = FAVOURITES_ID_COLUMN)
#RenameColumn(incorrect_table_name, fromColumnName = incorrect_table_name+"_desc", toColumnName = FAVOURITES_DESCRIPTION_COLUMN)
#RenameTable(fromTableName = incorrect_table_name, toTableName = FAVOURITES_TABLE_NAME)
class am2to3 : AutoMigrationSpec
along with :-
#Database(
entities = [Favourite::class],
version = DATABASE_VERSION,
autoMigrations = [
AutoMigration(2,3, TheDatabase.am2to3::class)
]
)
never changed as it never went to version 2
Results
Device 1 (API 30) :-
Device 2 (API 28) :-
i.e. now with favouritelist NOT favouriteslist (likewise column names changed accordingly)
Related
We have a requirement to set up Kafka MicrosoftSqlServerSource connect.
This is to capture all the transactions(insert/update) performing in one of the sales table in Azure SQL database.
In order to bring the support for the above source connect, we have initially enabled CDC at both database and table level.
We also created a view out of the source table which will be the input for the source connect( TableType = VIEW in connector configuration).
Once we complete the set up at both connector as well as database level, we could see messages flowing to the respective topic created automatically along with the connector as when a new updations/insertions happened at the table level.
One strange behavior we observed while testing is that when we stopped the testing, the last message received in the topic starts getting duplicated until a new message arrived.
Could you please help us to understand whether this is a system behavior?
Or Did we miss any configuration that has resulted in these duplicate entries.
Please guide us on how we can tackle the above duplicate issue.
Attaching the snapshot
Connector Summary
Connector Class = MicrosoftSqlServerSource
Max Tasks = 1
kafka.auth.mode = SERVICE_ACCOUNT
kafka.service.account.id = **********
topic.prefix = ***********
connection.host = **************8
connection.port = 1433
connection.user = ***************
db.name = **************88
table.whitelist = item_status_view
timestamp.column.name = ProcessedDateTime
incrementing.column.name = SalesandRefundItemStatusID
table.types = VIEW
schema.pattern = dbo
db.timezone = Europe/London
mode = timestamp+incrementing
timestamp.initial = -1
poll.interval.ms = 10000
batch.max.rows = 1000
timestamp.delay.interval.ms = 30000
output.data.format = JSON
What you're describing is controlled by
mode = timestamp+incrementing
poll.interval.ms = 10000
It should save the last timestamp, then query only for timestamps greater than the last... If you are getting greater than or equal to, then that is certainly a bug that should be reported.
Or you should read the docs
A timestamp column must use datetime2 and not datetime. If the timestamp column uses datetime, the topic may receive numerous duplicates
As an alternative, you could use Debezium (run your own Connector, not use Confluent Cloud offering) to truly stream all table operations.
I have the following code:
/** #test */
public function it_updates_customer_status_to_deactivated_for_admin_users()
{
$this->hubAdminUser = factory(User::class)->state('admin')->create();
$this->customer = Customer::first();
$this->customer->status_id = 2; //active
$this->customer->save();
// this will update status_id to 3
$this->actingAs($this->hubAdminUser)
->patch(route('hub.customer.updateStatus', $this->customer))
->assertRedirect();
$this->assertDatabaseHas('tenants', [
'id' => $this->customer->id,
'status_id' => 3, //deactivated
]);
}
The ->patch(route('hub.customer.updateStatus', $this->customer)) line will change the value of status_id from 2 to 3 which it definitely does as I have even tried $this->customer->refresh()->status_id after the ->assertRedirect(); line and that gives me 3. This is failing as it says that the customer's status_id is set to 2 in the database. Any ideas how I can fix this?
I would you recommend to change assertDatabaseHas to assertEquals.
Here is why:
"When you’re working with Eloquent, you specify a table name - or it automatically figures it out. Then, you forget about it. So, it’s ideally designed for you not to have to know the name of the table."
"Laravel is architected, then, so that we don’t have to know the names of our database tables. With assertDatabaseHas you have to know the name of the table every time you use it."
"But, I think it’s best to stop asserting directly against a database when you don’t need to. Especially since your code is not generally architected in Laravel to deal directly with the database, why would your tests? Stay in your domain and test the input and output values, not the implementation."
From a article https://www.aaronsaray.com/2020/stop-assert-database-has-laravel
Started using the 'Preview App' late last week.
In the classic UI, we can highlight multiple rows and execute them all, like this for example...
update table.blah set updated_ts = current_timestamp()::timestamp_ntz, value = 1 where account_id = 5;
update table.blah set updated_ts = current_timestamp()::timestamp_ntz, value = 1 where account_id = 6;
In the new app.snowflake UI, we get the error...
000006 (0A000): Multiple SQL statements in a single API call are not supported; use one API call per statement instead.
This is a major annoyance. Is this something that is planned to be changed in a future release?
Thanks,
Craig
This is one of the current limitations of the new UI for Snowflake. Remember, it is only in Preview. You will see more and more functionality from the old interface be moved into the new interface over time. I am sure this is one of them.
To give a simplified example:
I have a database with one table: names, which has 1 million records each containing a common boy or girl's name, and more added every day.
I have an application server that takes as input an http request from parents using my website 'Name Chooser' . With each request, I need to pick up a name from the db and return it, and then NOT give that name to another parent. The server is concurrent so can handle a high volume of requests, and yet have to respect "unique name per request" and still be high available.
What are the major components and strategies for an architecture of this use case?
From what I understand, you have two operations: Adding a name and Choosing a name.
I have couple of questions:
Qustion 1: Do parents choose names only or do they also add names?
Question 2 If they add names, doest that mean that when a name is added it should also be marked as already chosen?
Assuming that you don't want to make all name selection requests to wait for one another (by locking of queueing them):
One solution to resolve concurrency in case of choosing a name only is to use Optimistic offline lock.
The most common implementation to this is to add a version field to your table and increment this version when you mark a name as chosen. You will need DB support for this, but most databases offer a mechanism for this. MongoDB adds a version field to the documents by default. For a RDBMS (like SQL) you have to add this field yourself.
You havent specified what technology you are using, so I will give an example using pseudo code for an SQL DB. For MongoDB you can check how the DB makes these checks for you.
NameRecord {
id,
name,
parentID,
version,
isChosen,
function chooseForParent(parentID) {
if(this.isChosen){
throw Error/Exception;
}
this.parentID = parentID
this.isChosen = true;
this.version++;
}
}
NameRecordRepository {
function getByName(name) { ... }
function save(record) {
var oldVersion = record.version - 1;
var query = "UPDATE records SET .....
WHERE id = {record.id} AND version = {oldVersion}";
var rowsCount = db.execute(query);
if(rowsCount == 0) {
throw ConcurrencyViolation
}
}
}
// somewhere else in an object or module or whatever...
function chooseName(parentID, name) {
var record = NameRecordRepository.getByName(name);
record.chooseForParent(parentID);
NameRecordRepository.save(record);
}
Before whis object is saved to the DB a version comparison must be performed. SQL provides a way to execute a query based on some condition and return the row count of affected rows. In our case we check if the version in the Database is the same as the old one before update. If it's not, that means that someone else has updated the record.
In this simple case you can even remove the version field and use the isChosen flag in your SQL query like this:
var query = "UPDATE records SET .....
WHERE id = {record.id} AND isChosend = false";
When adding a new name to the database you will need a Unique constrant that will solve concurrenty issues.
I am creating a small database for 'trial', per se. I have tried a few setting outs ([{key: value}, {key: value}]. But, I need a solution that can be called by an ID (12345), a name (Rob Alsod), an area (A4 (Like an apartment building)), or a job (Manager, Administrator, etc). So, something like a dictionary (which can only be called by one key) will not work. I tried making a 'Person' class, but I need a way to easily keep track of the classes, and also to assign them easily. For example,
for whatever in whatever:
what = Person(name = 'Rob Alsod', id = 12345, job = 'Admin', area = 'A1') # What can I make this iterate with? (Badly formed question)
My point is, as it loops through, I cannot assign it to the same thing again and again.
Could someone try to make sense of what I am saying, and suggest a way to format my database?
You could easily use SQLAlchemy with SQLite. SQL queries are only a few lines of code away:
from sqlalchemy import *
db = create_engine('sqlite:///people.db')
metadata = MetaData()
user = Table('people', metadata,
Column('ID', Integer, primary_key = True),
Column('name', String(16), nullable = False),
Column('area', String(16)),
Column('job', String(60))
)
After that it is easy connection = db.connect(); result = connection.execute("select ID from people")
On Linux, pip and apt-get make the install a breeze as well.
HTH, Phil
If you want small database for storing any objects identified by key you can use shelve module:
http://docs.python.org/3.2/library/shelve.html