When Indexing the items, it fails sometimes and it gives,
The remote server returned an error: (504) Gateway Timeout. [The remote server returned an error: (504) Gateway Timeout.]
The Indexing logic is here as below,
var client = EPiServer.Find.Framework.SearchClient.Instance;
List<ItemModel> items = getItems(); // Get more than 1000 items
List<ItemModel> tempItems = new List<ItemModel>();
//Index 50 items at a time
foreach(var item in items)
{
tempItems.Add(item);
if (tempItems.Count == 50)
{
client.Index(tempItems);
tempItems.Clear();
}
}
What causes this to happen ?
Note: The above mentioned ItemModel is a custom model which is not implemented interfaces (such as IContent). And the items is a list of ItemModel objects.
Additional info:
EPiServer.Find.Framework version 13.0.1
EPiServer.CMS.Core version 11.9.2
I always figured the SearchClient to be a bit sketchy when manipulating data in Find, as far as I figured (but I have to check this) the SearchClient obey under the request limitation of Episerver Find and when doing bigger operations in loops it tends to time out.
Instead, use the ContentIndexer, i.e.
// Use this or injected parameter
var loader = ServiceLocator.Current.GetInstance<IContentLoader>();
// Remove all children or not
var cascade = true;
ContentReference entryPoint = ...where you want to start
// Get all indexable languages from Find
Languages languages = SearchClient.Instance.Settings.Languages;
// Remove all current instances of all languages below the selected content node
//languages.ForEach(x => ContentIndexer.Instance.RemoveFromIndex(entryPoint, cascade.Checked, x.FieldSuffix));
foreach (var lang in languages)
{
if (cascade)
{
var descendents = loader.GetDescendents(entryPoint);
foreach (ContentReference descendent in descendents)
{
ContentIndexer.Instance.RemoveFromIndex(descendent, false, lang.FieldSuffix);
}
}
// Try delete the entrypoint
var entryTest = loader.Get<IContent>(entryPoint, new CultureInfo(lang.FieldSuffix));
if (entryTest != null)
{
var delRes = ContentIndexer.Instance.Delete(entryTest);
}
}
This is the most bulletproof way to delete stuff from the index as far as I figured.
Related
hi i want to get follower list via tweetsharp but i have a exception this = Unable to cast object of type 'System.Collections.Generic.List1[TweetSharp.TwitterUser]' to type 'TweetSharp.TwitterCursorList1[TweetSharp.TwitterUser]'.
help me please HOW CAN I GET FOLLOWERLIST?
my code like this
string aranan = "anilsarii";
var AramaAyari = new SearchForUserOptions { Q = aranan, Count = 25 };
var users = ts.SearchForUser(AramaAyari); //Get list of users by query
//...
//var asd=ts.FollowList(new FollowListOptions{ OwnerId= 2603023494});
var followers = ts.ListFollowers(new ListFollowersOptions { Cursor = -1 });
while (followers.NextCursor != null)
{
followers = ts.ListFollowers(new ListFollowersOptions { followers.NextCursor });
}
You may just need to change this line;
followers = ts.ListFollowers(new ListFollowersOptions { followers.NextCursor });
to
followers = ts.ListFollowers(new ListFollowersOptions { Cursor = followers.NextCursor });
If that doesn't work, it's likely a problem in TweetSharp itself. TweetSharp has been abandoned by the original authors and is no longer maintained. As a result, there are a number of problems using it with the current Twitter API, due to changes on Twitter's end in the last few years. However there are several forks of TweetSharp that are current.
I tried your code with the one line modified above using TweetMoaSharp (a fork, available on Nuget, or on github here; https://github.com/Yortw/tweetmoasharp) and it worked fine.
Full disclosure: TweetMoaSharp is a fork primarily maintained by me. If you search Nuget for 'TweetSharp' you should find several others.
I have this many to many association between KundeInfo and HovedKategori, which I have mapped in my MS SQL database like this:
I have implemented the methods KundeInfo.HovedKategoris:
public IEnumerable<KundeInfo> KundeInfos
{
get
{
using (var dc = new DataClassesBSMAKSDataContext())
{
dc.DeferredLoadingEnabled = false;
var kundeInfoHovedKategoris = dc.KundeInfoHovedKategoris.Where(x => x.HovedKategori_Id == Id);
var kundeInfos = dc.KundeInfos.Where(x => kundeInfoHovedKategoris.Any(y => y.KundeInfo_Id == x.Id));
return kundeInfos.ToList();
}
}
}
... and HovedKategori.KundeInfos:
public IEnumerable<HovedKategori> HovedKategoris
{
get
{
using (var dc = new DataClassesBSMAKSDataContext())
{
var kundeInfoHovedKategoris = dc.KundeInfoHovedKategoris.Where(x => x.KundeInfo_Id == Id);
var hovedKategoris = dc.HovedKategoris.Where(x => kundeInfoHovedKategoris.Any(y => y.HovedKategori_Id == x.Id));
return hovedKategoris.ToList();
}
}
}
This retrieves the associated KundeInfos from a specific HovedKategori and opposite. The problemhowever lies in the serialization. When I call ToList(), or serialize these objects to JSON, linq will try to first follow all references returned by HovedKategori.KundeInfos, if it were that method I were to call first, and then it would for each returned object, try to follow all references returned by KundeInfo.HovedKategoris and so on, until it would cast a stack overflow exception.
If I could somehow prevent linq from following certain properties with an [Ignore] attribute or something, it would work, but I haven't been able to find anything like that.
What can I do in this situation?
This is in part a design issue. What you should really ask yourself is if you need navigation properties in every possible direction. For example if you just add a Kategori ID instead of a navigation property you could still query your context (by using the ID) but do you really need to always get all the Kategori's with all underlying data?
also if you make your properties virtual you have lazy loading and will only get the information if you .Include it or explicitly reference it.
Ok - So I solved this problem by just making it into methods on the respective classes, which it should be in the first place, since it retrieved these entities from the database. So yes, partially design issue.
Virtual did not work, I considered using projection and an abstract class, but it would be such a haystack of inheritance, and class casts, that it would not even be worth considering.
public IEnumerable<KundeInfo> KundeInfos()
{
using (var dc = new DataClassesBSMAKSDataContext())
{
var kundeInfoHovedKategoris = dc.KundeInfoHovedKategoris.Where(x => x.HovedKategori_Id == Id);
var kundeInfos = dc.KundeInfos.Where(x => kundeInfoHovedKategoris.Any(y => y.KundeInfo_Id == x.Id));
return kundeInfos.ToList();
}
}
You can get the child count via
firebase_node.once('value', function(snapshot) { alert('Count: ' + snapshot.numChildren()); });
But I believe this fetches the entire sub-tree of that node from the server. For huge lists, that seems RAM and latency intensive. Is there a way of getting the count (and/or a list of child names) without fetching the whole thing?
The code snippet you gave does indeed load the entire set of data and then counts it client-side, which can be very slow for large amounts of data.
Firebase doesn't currently have a way to count children without loading data, but we do plan to add it.
For now, one solution would be to maintain a counter of the number of children and update it every time you add a new child. You could use a transaction to count items, like in this code tracking upvodes:
var upvotesRef = new Firebase('https://docs-examples.firebaseio.com/android/saving-data/fireblog/posts/-JRHTHaIs-jNPLXOQivY/upvotes');
upvotesRef.transaction(function (current_value) {
return (current_value || 0) + 1;
});
For more info, see https://www.firebase.com/docs/transactions.html
UPDATE:
Firebase recently released Cloud Functions. With Cloud Functions, you don't need to create your own Server. You can simply write JavaScript functions and upload it to Firebase. Firebase will be responsible for triggering functions whenever an event occurs.
If you want to count upvotes for example, you should create a structure similar to this one:
{
"posts" : {
"-JRHTHaIs-jNPLXOQivY" : {
"upvotes_count":5,
"upvotes" : {
"userX" : true,
"userY" : true,
"userZ" : true,
...
}
}
}
}
And then write a javascript function to increase the upvotes_count when there is a new write to the upvotes node.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.countlikes = functions.database.ref('/posts/$postid/upvotes').onWrite(event => {
return event.data.ref.parent.child('upvotes_count').set(event.data.numChildren());
});
You can read the Documentation to know how to Get Started with Cloud Functions.
Also, another example of counting posts is here:
https://github.com/firebase/functions-samples/blob/master/child-count/functions/index.js
Update January 2018
The firebase docs have changed so instead of event we now have change and context.
The given example throws an error complaining that event.data is undefined. This pattern seems to work better:
exports.countPrescriptions = functions.database.ref(`/prescriptions`).onWrite((change, context) => {
const data = change.after.val();
const count = Object.keys(data).length;
return change.after.ref.child('_count').set(count);
});
```
This is a little late in the game as several others have already answered nicely, but I'll share how I might implement it.
This hinges on the fact that the Firebase REST API offers a shallow=true parameter.
Assume you have a post object and each one can have a number of comments:
{
"posts": {
"$postKey": {
"comments": {
...
}
}
}
}
You obviously don't want to fetch all of the comments, just the number of comments.
Assuming you have the key for a post, you can send a GET request to
https://yourapp.firebaseio.com/posts/[the post key]/comments?shallow=true.
This will return an object of key-value pairs, where each key is the key of a comment and its value is true:
{
"comment1key": true,
"comment2key": true,
...,
"comment9999key": true
}
The size of this response is much smaller than requesting the equivalent data, and now you can calculate the number of keys in the response to find your value (e.g. commentCount = Object.keys(result).length).
This may not completely solve your problem, as you are still calculating the number of keys returned, and you can't necessarily subscribe to the value as it changes, but it does greatly reduce the size of the returned data without requiring any changes to your schema.
Save the count as you go - and use validation to enforce it. I hacked this together - for keeping a count of unique votes and counts which keeps coming up!. But this time I have tested my suggestion! (notwithstanding cut/paste errors!).
The 'trick' here is to use the node priority to as the vote count...
The data is:
vote/$issueBeingVotedOn/user/$uniqueIdOfVoter = thisVotesCount, priority=thisVotesCount
vote/$issueBeingVotedOn/count = 'user/'+$idOfLastVoter, priority=CountofLastVote
,"vote": {
".read" : true
,".write" : true
,"$issue" : {
"user" : {
"$user" : {
".validate" : "!data.exists() &&
newData.val()==data.parent().parent().child('count').getPriority()+1 &&
newData.val()==newData.GetPriority()"
user can only vote once && count must be one higher than current count && data value must be same as priority.
}
}
,"count" : {
".validate" : "data.parent().child(newData.val()).val()==newData.getPriority() &&
newData.getPriority()==data.getPriority()+1 "
}
count (last voter really) - vote must exist and its count equal newcount, && newcount (priority) can only go up by one.
}
}
Test script to add 10 votes by different users (for this example, id's faked, should user auth.uid in production). Count down by (i--) 10 to see validation fail.
<script src='https://cdn.firebase.com/v0/firebase.js'></script>
<script>
window.fb = new Firebase('https:...vote/iss1/');
window.fb.child('count').once('value', function (dss) {
votes = dss.getPriority();
for (var i=1;i<10;i++) vote(dss,i+votes);
} );
function vote(dss,count)
{
var user='user/zz' + count; // replace with auth.id or whatever
window.fb.child(user).setWithPriority(count,count);
window.fb.child('count').setWithPriority(user,count);
}
</script>
The 'risk' here is that a vote is cast, but the count not updated (haking or script failure). This is why the votes have a unique 'priority' - the script should really start by ensuring that there is no vote with priority higher than the current count, if there is it should complete that transaction before doing its own - get your clients to clean up for you :)
The count needs to be initialised with a priority before you start - forge doesn't let you do this, so a stub script is needed (before the validation is active!).
write a cloud function to and update the node count.
// below function to get the given node count.
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.userscount = functions.database.ref('/users/')
.onWrite(event => {
console.log('users number : ', event.data.numChildren());
return event.data.ref.parent.child('count/users').set(event.data.numChildren());
});
Refer :https://firebase.google.com/docs/functions/database-events
root--|
|-users ( this node contains all users list)
|
|-count
|-userscount :
(this node added dynamically by cloud function with the user count)
I'd like to be able to search model attributes contained within a backbonejs collection. This is how I do it now...
wherePartial: function(attrs) {
// this method is really only tolerant of string values. you can't do partial
// matches on objects, but you can compare a list of strings. If you send it a list
// of values; attrs={keyA:[1,2,3],keyB:1}, etc the code will loop through the entire
// attrs obj and look for a match. strings are partially matched and if a list is found
// it's expected that it contains a list of string values. The string values should be considered
// to be like an OR operation in a query. Non-list items are like an AND.
if (_.isEmpty(attrs)) return [];
var matchFound = false;
return this.filter(function(model) {
// this is in the outer for loop so that a function isn't created on each iteration
function listComparator(value, index, list){
return model.get(key).toLowerCase().indexOf(value.toLowerCase()) >= 0;
}
for (var key in attrs) {
if (_.isArray(attrs[key])){
matchFound = _.any(attrs[key],listComparator);
if (matchFound !== true) return false;
} else {
matchFound = model.get(key).toLowerCase().indexOf(attrs[key].toLowerCase()) >= 0;
if (matchFound === false) return false;
}
}
return true;
});
}
Assume "C" is an instantiated collection, this is how I use it:
name:joe (nickname:joe the man nickname:joe cool nickname:joey)
is typed into a textbox and converted into this:
C.wherePartial({name:"joe",nicknames:["joe the man","joe cool","joey"]})
The above method returns all models that have the name joe and within that scope, any of the models that have the name joe and any of the nicknames. It works well for what I use it for. However, I'd really like to make a search that doesn't require the key:value pattern. I'd like to do this in a search box like when using a search engine on the web. I thought about just looking at every attribute on each model, but that takes awhile when you have a large collection (160k+ models).
Has anyone come across a need like this in the past? If so, how did you solve it? I'd like to keep the search contained on the client and not use any ajax calls to the backend. The reason for this is that the entire collection is already loaded on the client.
I thought of a way to do it. Serialize the attributes to a string during model instantiation. Listen for updates and update the serialization.
serializeAttr: function(){
this.serializedAttr = "";
var self = this;
_.each(this.toJSON(),function(value, key, list){
self.serializedAttr += value;
});
}
Then I can do simple searches on that cached value:
cc.serializedAttr.toLowerCase().indexOf("joe") >= 0
I am using SL 4, WCF RIA Services against Entity Framework 4.0. I have an Entity, Visit, that has a string Status field. I have a search screen where I need to display results that have StatusA or StatusB. I am struggling to find a way to specify a client-side query that specifies a collection of statuses that should be matched. If I was to write what I want in SQL it would look something like:
select * from Visit where Status in ('StatusA', 'StatusB');
Client side, it appears to be straightforward to chain Where methods for a WhereAnd effect:
var query = this.PqContext.GetVisitsQuery();
if (!string.IsNullOrEmpty(this.PracticeName))
{
query = query.Where(v => v.PracticeName.ToUpper().Contains(this.PracticeName.ToUpper()));
}
if (this.VisitDateAfter.HasValue)
{
query = query.Where(v => v.VisitDate > this.VisitDateAfter);
}
if (this.VisitDateBefore.HasValue)
{
query = query.Where(v => v.VisitDate < this.VisitDateBefore);
}
However, I can't seem to find a straightforward way to do a WhereOr style operation. I have tried this:
var statuses = new List<string>();
if (this.ShowStatusA)
{
statuses.Add("StatusA");
}
if (this.ShowStatusB)
{
statuses.Add("StatusB");
}
if (statuses.Any())
{
query = query.Where(BuildContainsExpression<Visit, string>(e => e.Status, statuses));
}
Where BuildContainsExpression looks like:
private static Expression<Func<TElement, bool>> BuildContainsExpression<TElement, TValue>(Expression<Func<TElement, TValue>> valueSelector, IEnumerable<TValue> values)
{
if (null == valueSelector)
{
throw new ArgumentNullException("valueSelector");
}
if (null == values)
{
throw new ArgumentNullException("values");
}
ParameterExpression p = valueSelector.Parameters.Single();
if (!values.Any())
{
return e => false;
}
var equals =
values.Select(
value =>
(Expression)Expression.Equal(valueSelector.Body, Expression.Constant(value, typeof(TValue))));
var body = equals.Aggregate<Expression>(Expression.Or);
return Expression.Lambda<Func<TElement, bool>>(body, p);
}
But this throws a "Bitwise operators are not supported in queries." exception. Any clues? Is there an alternative way to build an expression tree that works here or do I need to pass all the parameters over to the server and use the BuildContainsExpression there?
Your time and your guidance are much appreciated.
You can create a query method such as the following in your domain service:
GetVisitsByStatus(string[] statusList) {
// create the LINQ where clause here
}
And then from the client, call context.GetVistsByStatusQuery(string[]).
Not all of LINQ is (or even can) be exposed over the URL, so there are always cases where you need to use simple parameters, and have the middle tier construct the LINQ expressions that eventually define the query that goes to the back-end data store.
Hope that helps.