How should one configure Slick to persist tables between sessions? - database

I have experienced some issues while setting up Slick 2.0.2. Any configuration that I do in one session is lost in the next. For example, in the first session, I create the table and add three people:
// H2 in-memory database
lazy val db = Database.forURL("jdbc:h2:mem:contacts", driver="org.h2.Driver")
// Contacts table
lazy val contacts = TableQuery[ContactsSchema]
// Initial session
db withSession { implicit session =>
contacts.ddl.create
// Inserts sample data
contacts += Person("John", "123 Main street", 29)
contacts += Person("Greg", "Neither here nor there", 40)
contacts += Person("Michael", "Continental U.S.", 34)
// Successfully retrieves data
contacts foreach { person =>
println(person)
}
}
All is well up to this point. The output repeats the three people whom I added. When I start a new session, I start to experience issues.
// New session in which the previous data is lost
db withSession { implicit session =>
contacts foreach { person =>
println(person)
}
}
The above block creates a org.h2.jdbc.JdbcSQLException: Table "CONTACTS" not found exception. If I edit as follows
db withSession { implicit session =>
contacts.ddl.create
contacts foreach { person =>
println(person)
}
}
then all the data is erased.
I see that the Scalatra guide to Slick uses a similar configuration to mine. What am I doing wrong? How should I get the data to persist between sessions? Does the fact that I am using an in-memory database have anything to do with it?

Two choices.
Either create a session and keep it open. That can be done with a withSession scope lower on the call stack or db.createSession.
Or add ;DB_CLOSE_DELAY=-1 to the database url. That keeps the db alive as long as the vm runs.
See http://www.h2database.com/html/features.html#in_memory_databases

Related

Entity Framework doesn't alter database

I'm new to Entity frameworks and got some questions. I've got an EFcore project where there needs to be a database around users, with roles, groups,..
First I had Razor CRUD pages where everything worked. I could add, update and delete users, roles, groups,... But along the way I realised that I rahter needed a SwaggerUI so I could use that API for an other frontend project.
So I changed the razor pages to Swagger and for some reason the database doesn't change when I Update, delete or post something. Without any warnings. I even get succes codes back as feedback.
But the action doesn't really go through.
(When I delete, it says deteled but record is still the same. Same with Update and with a post, it says that the creation succeeded but the new record is not in my database.
I can view all records with Get & specific Records with Get:ID so I'm kind of lost why my update, post or delete action don't work.
I'm kind of new in this area so any feedback is much appreciated.
Thanks in advance.
Try to update/delete/post a record in db. Always gives
UserController : ( this worked with the Razor pages but not with the swagger page)
[HttpDelete("{id}")]
public ActionResult<User> Delete(int id)
{
var user = _userRepo.Get(id);
if(user != null)
{
_userRepo.Delete(id);Console.WriteLine("is deleted.");
}
else
{
return NotFound();
}
return NoContent();
}
[HttpPost()]
public IActionResult Add([FromBody] UserCreateViewModel model)
{
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
var newUser = new User
{
FirstName = model.FirstName,
LastName = model.LastName,
Email = model.Email,
Platform = model.Platform,
Is_enabled = model.Is_enabled,
};
Console.WriteLine(newUser);
_userRepo.Add(newUser);
return CreatedAtAction(nameof(Get), new { newUser.Id }, newUser);
}
Note: The Console.WriteLine("is deleted."); does run and is shown in the console. But it doesn't delete the record.
In Entity Framework, the SaveChanges() method internally creates a
transaction and wraps all INSERT, UPDATE and DELETE operations under
it. Multiple SaveChanges() calls, create separate transactions,
perform CRUD operations and then commit each transaction.
https://www.entityframeworktutorial.net/entityframework6/transaction-in-entity-framework.aspx#:~:text=In%20Entity%20Framework%2C%20the%20SaveChanges,and%20then%20commit%20each%20transaction.

Journey builder's custom activity: Fetch data extension data in bulk

I am new to Salesforce Marketing Cloud and journey builder.
https://developer.salesforce.com/docs/marketing/marketing-cloud/guide/creating-activities.html
We are building journey builder's custom activity in which it will use a data extension as the source and when the journey builder is invoked, it will fetch a row and send this data to our company's internal endpoint. The team got that part working. We are using the postmonger.js.
I have a couple of questions:
Is there a way to retrieve the data from data extension in bulk so that we can call our company's internal bulk endpoint? Calling the endpoint for each record in the data extension for our use case would not be efficient enough and won't work.
When the journey is invoked and an entry in the data extension is retrieved and that data is sent to our internal endpoint, is there a machanism to mark this entry as already sent such that next time the journey is run, it won't process the entry that's already sent?
Here is a snippet of our customActivity.js in which this is populating one record. (I changed some variable names.). Is there a way to populate multiple records such that when "execute" is called, it is passing a list of payloads as input to our internal endpoint.
function save() {
try {
var TemplateNameValue = $('#TemplateName').val();
var TemplateIDValue = $('#TemplateID').val();
let auth = "{{Contact.Attribute.Authorization.Value}}"
payload['arguments'].execute.inArguments = [{
"vendorTemplateId": TemplateIDValue,
"field1": "{{Contact.Attribute.DD.field1}}",
"eventType": TemplateNameValue,
"field2": "{{Contact.Attribute.DD.field2}}",
"field3": "{{Contact.Attribute.DD.field3}}",
"field4": "{{Contact.Attribute.DD.field4}}",
"field5": "{{Contact.Attribute.DD.field5}}",
"field6": "{{Contact.Attribute.DD.field6}}",
"field7": "{{Contact.Attribute.DD.field7}}",
"messageMetadata" : {}
}];
payload['arguments'].execute.headers = `{"Authorization":"${auth}"}`;
payload['configurationArguments'].stop.headers = `{"Authorization":"default"}`;
payload['configurationArguments'].validate.headers = `{"Authorization":"default"}`;
payload['configurationArguments'].publish.headers = `{"Authorization":"default"}`;
payload['configurationArguments'].save.headers = `{"Authorization":"default"}`;
payload['metaData'].isConfigured = true;
console.log(payload);
connection.trigger('updateActivity', payload);
} catch(err) {
document.getElementById("error").style.display = "block";
document.getElementById("error").innerHtml = err;
}
console.log("Template Name: " + JSON.stringify(TemplateNameValue));
console.log("Template ID: " + JSON.stringify(TemplateIDValue));
}
});
Any advise or idea is highly appreciated!
Thank you.
Grace
Firstly, i implore you to not proceed with the design pattern of fetching data for each subscriber, from Marketing Cloud, that gets sent through the custom activity, for arguments sake i'll list two big issues.
You have no way of limiting the configuration of data extensions columns or column names in SFMC (Salesforce Marketing Cloud). If any malicious user or by human error would delete a column or change a column name your service would stop receiving that value.
Secondly, Marketing Cloud has 2 sets of API limitations, yearly and minute by minute. Depending on your licensing, you could run into the yearly limit.
The problem you have with limitation on minutes (2500 for REST and 2000 for SOAP) is that each usage of the custom activity in journey builder would multiple the amount of invocations per minute. Hitting this limit would cause issues for incremental data flows into SFMC.
I'd also suggest not retrieving any data from Marketing Cloud when a customer gets sent through a custom activity. Users should pick which corresponding rows/data that should be sent to the custom activity in their segmentation.
The eventDefinitionKey can be picked up from postmonger after requestedTriggerEventDefinition in the eventDefinitionModel function. eventDefinitionKey can then be used to programmatically populate SFMC's POST call with data from the Journey Data model, thus allowing marketers to select what data to be sent with the subscriber.
Following is some code to show how it would work in your customActivity.js
connection.on(
'requestedTriggerEventDefinition',
function (eventDefinitionModel) {
var eventKey = eventDefinitionModel['eventDefinitionKey'];
save(eventKey);
}
);
function save(eventKey) {
// subscriberKey fetched directly from Contact model
// columnName is populated from the Journey Data model
var params = {
subscriberKey: '{{Contact.key}}',
columnName: '{{Event.' + eventKey + '.columnName}}',
};
payload['arguments'].execute.inArguments = [params];
}

How to restrict files in a database/table to certain users logged into wix website

I have recently created a website on Wix that requires you to log in to access the "Client Portal" member page I created. The client portal page is essentially a database (collection) in table form that allows users to select and view documents pertaining to their company. The ClientPortal2 collection has 3 fields: the name of the document, the actual document, and the company it is affiliated with. The Client Portal database is shown below. I want to restrict users to only be able to see the documents that are affiliated with their company so I made another collection called Users2. Users2 references all of the users that have an account in a multi reference field (email) and the company that they work for in another field (company). Users2 is also shown below. In summary, I want users to ONLY be able to look at their company's documents and no one else's.
CLIENTPORTAL2
Users2
The code I wrote to sort the documents based on the certain user's affiliated company is as shown below:
import wixUsers from 'wix-users';
import wixData from 'wix-data';
$w.onReady(function () {
let user = wixUsers.currentUser;
user.getEmail()
.then( (currentEmail) => {
let userEmail = currentEmail; // "user#something.com"
} );
user.getEmail()
.then( (currentEmail) => {
console.log(currentEmail);
wixData.query('Users2').include('email')
.find().then((results) => {
for (var i=0;i<results.items.length ;i++) {
for (var j=0;j<results.items[i].email.length ;j++) {
let userLoginEmail = results.items[i].email[j].loginEmail;
if (userLoginEmail === currentEmail){
let userCompany = results.items[i].company
filter(userCompany);
}
}
}
})
})
});
function filter(Company){
wixData.query('CLIENTPORTAL2').eq("company", Company)
.find().then( (results) => {
let tableData = results.items;
$w('#table1').rows = tableData;
console.log(results.items);
console.log(Company);
})
}
The code runs a query on 'Users2' and looks at all of the users' email addresses and compares it to the current logged in user's address. It then runs a query on 'ClientPortal2' using the users affiliated company it just found in the first query. The table (#table1) displayed on the Client Portal page now displays the documents pertaining to the company the user works for. So for example, if 'dew0025#auburn.edu' was logged into the site, that user would only be able to view the documents in the table below because that user is restricted to 'Company 1'
table1 displayed on Client Portal Members Page
So this code works well but it only works on the preview page and not the live published page. What am I doing wrong? Is there an easier way of doing this or am I going about this the wrong way? Thanks in advance.
Preview mode uses the Sandbox database and live sites use the live database. If you insert an item in the sandbox database it will not appear in the live database unless you sync your databases.
Also check for permissions on the database.

Unreliable Google Firebase transactions

In my (greatly simplified) model I have users, accounts and account_types. Each user can have multiple accounts of each account_type. When an account of type TT is created I'm updating the "users" field of that object so it keeps the users which have accounts of that types, and the number of such accounts they have.
users: {
some fields
},
accounts: {
userID: UU,
type: TT
},
account_type:
users: { UU: 31 }
}
I use the onCreate and onDelete cloud triggers for accounts to update the account_type object. Since multiple accounts can be created simultaneously I have to use transactions:
exports.onCreateAccount = functions.firestore
.document('accounts/{accountID}')
.onCreate((account, context) => {
const acc_user = account.data().userID;
const acc_type = account.data().type;
return admin.firestore().runTransaction(transaction => {
// This code may get re-run multiple times if there are conflicts.
const accountTypeRef = admin.firestore().doc("account_types/"+acc_type);
return transaction.get(accountTypeRef).then(accTypeDoc => {
var users = accTypeDoc.data().users;
if (users === undefined) {
users = {};
}
if (users[acc_user] === undefined) {
users[acc_user] = 1;
} else {
users[acc_user]++;
}
transaction.update(accountTypeRef, {users: users});
return;
})
})
.catch(error => {
console.log("AccountType create transaction failed. Error: "+error);
});
});
In my tests I'm first populating the database with some data so I'm also adding a user and 30 accounts of the same type. With the local emulator this works just fine and at the end of the addition I see that the account_type object contains the user with the counter at 30. But when deployed to Firebase and running the same functions the counter gets to less than 30. My suspicion is that since Firebase is much slower and transactions take longer, more of them are conflicted and fail and eventually don't execute at all. The transaction failure documentation (https://firebase.google.com/docs/firestore/manage-data/transactions) says:
"The transaction read a document that was modified outside of the transaction. In this case, the transaction automatically runs again. The transaction is retried a finite number of times."
So my questions:
What does "finite" mean?
Any way to control this number?
How can I make sure my transactions are executed at some point and don't get dropped like that so my data is consistent?
Any other idea as to why I'm not getting the correct results when deployed to the cloud?
What does "finite" mean?
It's the opposite of "unlimited". It will retry no more than a set number of times.
Any way to control this number?
Other than modifying the source code of the SDK, no. The SDK itself advertise a specific number, as it might change.
How can I make sure my transactions are executed at some point and don't get dropped like that so my data is consistent?
Detect the error and retry in your app. If you aren't seeing the transaction fail with an error, then nothing went wrong.
Any other idea as to why I'm not getting the correct results when deployed to the cloud?
Since we can't see what exactly you're doing to trigger the function, and have no specific expected results to compare to, it's not really possible to say.

Yii Dynamic DB Connection according to user with master database

My project is on the basis of multi-tenent SaaS.
I have multiple clients (companies) and each client has multiple users - they all will use the same database layout.
Each client has their own database, so during user authentication, I want to Build a master database that associates the user with a company database for that user.
The structure of each database is identical... only the data is different.
So that we can keep the different database for the different company, that will not going to mix in data in database.
The number of clients (and therefor the number of databases) is unknown when the application is written, so it is not possible to include all the connections in the bootstrap script.
Now, what I want to do is, dynamically alter the DB connection that is in the bootstrap or have the ability to dynamically create a new connection for the user signing in. Is there a simple solution for this in Yii and still use AR , query builder ?
I saw this solution but not working for me http://www.yiiframework.com/forum/index.php?/topic/5385-dynamic-db-connection/
This is how my config file looks today for the script running one database, i want to call a master database that controls which database the user belongs to and the app uses in Yii, any idea?
<? php
$language = 'en';
$currencyBaseCode = 'USD';
$theme = 'default';
$connectionString = 'mysql:host=localhost;port=3306;dbname=master';
$username = 'root';
$password = 'YOUR PASS';
$memcacheServers = array( // An empty array means memcache is not used.
array(
'host' => '127.0.0.1',
'port' => 11211, // This is the default memcached port.
'weight' => 100,
),
);
$adminEmail = 'EMAIL ADDRESS';
$installed = true; // Set to true by the installation process.
$maintenanceMode = false; // Set to true during upgrade process or other maintenance tasks.
$instanceConfig = array(); //Set any parameters you want to have merged into configuration array.
//#see CustomManagement
$instanceConfig['components']['request']['hostInfo'] = 'website url';
$instanceConfig['components']['request']['scriptUrl'] = '/app/index.php';
$urlManager = array (); // Set any parameters you want to customize url manager.
? >
Define the master database in the config file, and use the normal AR class to connect to it to retrieve the credentials. Then, close the connection.
Then extend the AR class, specifically the getDbConnection() method. Get the relevant credentials, and pass them as a parameters to a new CDbConnection object.
The initial DB request to get the credentials is going to be a huge performance drag, so you should really store them in-memory with Redis or Memcached. Yii doesn't do that natively.
There are a few different ways around this:
Different front controllers loading different config files, each of which is dynamically generated or destroyed by a separate app.
One big database. Each record has a key to define which client it belongs to.
One big database. Tables can be duplicated by using a client name as part of the table name.
None of these will be easy with Yii, as this is not really something Yii was designed to do.

Resources