I've recently started using the sql version of IDistributedCache on a dotnet core web api.
How would you remove/invalidate a set of keys for say a specific user?
I.e: I structured the keys to follow this convention:
/users/{userId}/Key1
/users/{userId}/Key2
/users/{userId}/Section/Key3
I cannot find any method to remove all keys starting with: /users/{userId}
How do you remove more than one item from the IDistributedCache at a time?
Removing via SQL statement is not a good solution because the webapp process performs some kind of lock. For example, I had to manually stop after three minutes and half the following simple query Delete from SqlSession with only two records.
So I ended up this way: I retrieve data with a simple query like
Select Id from SqlSession where Id like 'MyIdGroup%'
or with Entity framework
var cacheElementsToDelete = await _dbContext.SQLSessions
.Where(a => a.Id.StartsWith("MyIdGroup"))
.ToListAsync();
Then I use the method of the IDistributedCache to remove each item
foreach (var item in cacheElementsToDelete)
{
await _cache.RemoveAsync(item.Id);
}
Related
I working on an application that has its own database and gets user information from another serivce (an LDAP is this case, through an API package).
Say I have a tables called Articles, with a column user_id. There is no Users table, instead a user or set of users is retrieved through the external API:
$user = LDAPConnector::getUser($user_id);
$users = LDAPConnector::getUsers([1, 2, 5, 6]);
Of course I want retrieving data from inside a controller to be as simple as possible, ideally still with something like:
$articles = $this->Articles->find()->contain('Users');
foreach ($articles as $article) {
echo $article->user->getFullname();
}
I'm not sure how to approach this.
Where should I place the code in the table object to allow integration with the external API?
And as a bonus question: How to minimise the number of LDAP queries when filling the Entities?
i.e. it seems to be a lot faster by first retrieving the relevant users with a single ->getUsers() and placing them later, even though iterating over the articles and using multiple ->getUser() might be simpler.
The most simple solution would be to use a result formatter to fetch and inject the external data.
The more sophisticated solution would a custom association, and a custom association loader, but given how database-centric associations are, you'd probably also have to come up with a table and possibly a query implementation that handles your LDAP datasource. While it would be rather simple to move this into a custom association, containing the association will look up a matching table, cause the schema to be inspected, etc.
So I'll stick with providing an example for the first option. A result formatter would be pretty simple, something like this:
$this->Articles
->find()
->formatResults(function (\Cake\Collection\CollectionInterface $results) {
$userIds = array_unique($results->extract('user_id')->toArray());
$users = LDAPConnector::getUsers($userIds);
$usersMap = collection($users)->indexBy('id')->toArray();
return $results
->map(function ($article) use ($usersMap) {
if (isset($usersMap[$article['user_id']])) {
$article['user'] = $usersMap[$article['user_id']];
}
return $article;
});
});
The example makes the assumption that the data returned from LDAPConnector::getUsers() is a collection of associative arrays, with an id key that matches the user id. You'd have to adapt this accordingly, depending on what exactly LDAPConnector::getUsers() returns.
That aside, the example should be rather self-explanatory, first obtain a unique list of users IDs found in the queried articles, obtain the LDAP users using those IDs, then inject the users into the articles.
If you wanted to have entities in your results, then create entities from the user data, for example like this:
$userData = $usersMap[$article['user_id']];
$article['user'] = new \App\Model\Entity\User($userData);
For better reusability, put the formatter in a custom finder. In your ArticlesTable class:
public function findWithUsers(\Cake\ORM\Query $query, array $options)
{
return $query->formatResults(/* ... */);
}
Then you can just do $this->Articles->find('withUsers'), just as simple as containing.
See also
Cookbook > Database Access & ORM > Query Builder > Adding Calculated Fields
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Custom Finder Methods
I am currently pulling a list from a database, using the following code. The list is retrieved using a WHERE condition, however the list is returned unsorted. This is in the controller.
How can I modify this code so that the returned list is sorted alphabetically?
if (!string.IsNullOrEmpty(TargetYear))
{
ViewBag.HSID = new SelectList(db.Hotspots.Where(g => g.HSID.Contains(TargetYear)).ToList(), "ID", "HSID");
}
On several other fields I have used the following method to order, but I'm not sure how, or if I can combine this with the where clause above. The key piece is ".OrderBy(e=>e.FIELD), however this is precisely the piece I'm not sure how to integrate with the query.
ViewBag.LocalityCode = new SelectList(db.Localities.OrderBy(e=>e.LOCALITY1), "LOC_CODE", "LOCALITY1");
Other helpful bits of info:
ASP.Net MVC5
Microsoft SQL 2012
if (!string.IsNullOrEmpty(TargetYear))
{
var data =
db.Hotspots
.Where(g => g.HSID.Contains(TargetYear))
.OrderBy(e=>e.HSID)
.ToList();
ViewBag.HSID = new SelectList(data,"ID", "HSID");
}
I have a many-to-many structure mapped to entity framework. This is a sample of what it looks like:
User UserTag Tag
------- -------- -------
IdUser(PK) IdUserTag(PK) IdTag(PK)
Name IdUser(FK) TagName
Desc IdTag(FK) Active
Now, I needed to exclude from any request of any method the viewing of Tags that were Active=false.
First, I tried doing it manually in every method, like:
public User GetById(int id)
{
var item = UserRepository.GetById(id); //This is just a repository that calls the EF context
//EF automatically maps it to the *UserTags* property
foreach(var tag in item.UserTags)
{
if(tag.Tag.Active == false)
item.UserTags.Remove(tag);
}
}
But it throws the following exception:
The relationship could not be changed because one or more of the foreign-key properties is non-nullable
So, I wanted to know if there's a way to conditionaly filter every request made to a specific table, whether it is select or a join request.
Try this in your GetById method:
var user.UserTags = dbContext.Entry(user)
.Collection(u => u.UserTags)
.Query()
.Where(ut => ut.Active == true)
.ToList();
The supplied code fails because it is attempting to remove items from the data entities not the list. If you want to pass the data entity around instead of the data model, you need to not use Remove. Something like the below (untested should work).
tags = item.UserTags.Where((ut) => ut.Active).ToList();
This line will get you a list of data entities that are active. However, you should really map all of this into a data model (see AutoMapper) and then you would not be removing items from the database.
I am using SQLAlchemy and try to manage a model "Media" which has a many-to-one relationship with a "Booking". Is it safe to call scoped_session.delete() from within a before_commit event?
def before_commit(session):
r""" Invokes the ``before_commit`` method on all items in the session.
This allows the models to perform an update-action depending on their
new data. """
for item in session.deleted:
if hasattr(item, 'before_commit'):
item.before_commit(session, 'deleted')
for item in session.dirty:
if hasattr(item, 'before_commit'):
item.before_commit(session, 'dirty')
for item in session.new:
if hasattr(item, 'before_commit'):
item.before_commit(session, 'new')
event.listen(db.session.__class__, 'before_commit', before_commit)
class Booking(db.Model):
# ...
media = db.relationship(Media, backref='booking')
def before_commit(self, session, status):
r""" Validates the booking's data. If the booking is being deleted,
all its media will be deleted with it. """
if status == 'deleted':
# Delete all the media that is associated with this booking.
for media in self.media:
session.delete(media)
a mass delete() using either session.execute("delete...") or session.query(cls).delete() should be fine, it just emits that SQL on the current connection.
As far as session.delete(obj), it looks like before_commit() is invoked before the final flush(), so in that sense you can treat it like a before_flush() event. Try it out and you should see the DELETE being emitted, and if so then you're fine.
I have built classified website using Yii php framework. Now it is getting a lot of traffic. So I want to using caching to optimize the performance of the website.
There are two controllers I want to optimize.
One is the thread list controller: (example) http://www.shichengbbs.com/category/view/id/15
The other one the the thread controller: (example) http://www.shichengbbs.com/info/view/id/67900
What I have done:
the thread list is cached for 3mins.(The other option is update the thread list only when new thread comes)
set the last-modified time HTTP header for the thread view. (expire time is not set, as some user complain that the page appears unchanged after editing)
Partial caching the categories navigation fragment.(It appears on the left side of every page)
Use htaccess to set expire header for img/html/css/js.
Considered database sql caching for the thread list, but not done. As I thought it is the same as 1.
What else can I do to improve the website performance?
I assume you have done the Performance Tuning guide point 1 and 3. It's really helpful.
For number 2 you can use the CHttpCacheFilter
class CategoryController extends Controller {
private $_categoryLastUpdate;
public function filters(){
return array(
array(
'CHttpCacheFilter + view',
'cacheControl' => " max-age=604800, must-revalidate",
'etagSeedExpression' => function() {
return $this->getCategoryLastUpdate();
}
'lastModifiedExpression' => function() {
return $this->getCategoryLastUpdate();
}
)
)
}
public function actionView($id){
$object = Category::model()->findByPk($_GET['id']);
$this->render('view', array('object' => $object));
}
public function getCategoryLastUpdate(){
if (!isset($this->_categoryLastUpdate)){
$obj = Category::model()->findByPk($_GET['id'], array('select' => 'lastUpdate'));
$this->_categoryLastUpdate
}
return $this->_categoryLastUpdate;
}
}
It basically will calculate the ETag and LastUpdate by the category. And to save the query, it will first only calculate the lastUpdate of the Category object.
And for number one, you can always use the CCacheDependency. Just make a field in the thread list object, e.g. lastUpdate. And when a new thread submitted, just update the field and use it for the CCacheDependency.
Since I see you are using a very large pagination, I think you want to read about Four Ways to Optimize Paginated Displays (if you use MySQL for your database and thread search/list).
Try using a Cache Manager with Memcache or APC. For example, http://code.google.com/p/memcache-flag/ . When you edit the list, then you can invalidate the cache item or tag. I suppose it could also just be done with regular APC / Memcache functions if you design is simple (set a key and delete it when it is no longer valid).
Use this to store serialized (or automatically serialized) data instead of retrieving it from mysql.