I am trying to export the results of a user's search. I am using Django + Haystack + Solr to produce the search results. Currently, to create the SearchQuerySet to write out the CSV, I am passing the query parameters from the search results page to the view that produces the CSV and rebuilding the SearchQuerySet there. This is a real pain because the search is pretty complicated, with facets, multiple models, etc, and there are constantly bugs when I make modifications to the SearchForm. It seems like there should be an easy way to just pass the results directly to the export view. Any suggestions?
EDIT
I figured out my own solution and put all the modified code in the answer. Please see below. Hopefully this prevents someone else from banging their head against the wall for a week!
Okay, I finally figured this out myself. I basically had to add a second submit button to the html for my SearchForm and then used javascript to redirect the action to my "search_export" view. Since facets aren't passed along when the form is submitted, I had to get the facets from the request (in the search page template) and pass those to view rather hackishly through the url. The facets then have to be re-evaluated in the view. I will paste all of my code below:
search.html
{% block content %}
<form method="get" action=".">
<!-- Advanced Search Box -->
<div class="row">
<div class="col-lg-12">
<h2 class="text-center">Advanced Search</h2>
<!-- Search Form Fields Here -->
<ul class="list-inline center-block text-center adv-form">
<li>
<p><input type="submit" value="Search"></p>
</li>
<li>
<p><input type="submit" id="export" value="Export Results"></p>
</li>
</ul>
</div>
</div>
<!-- End Advanced Search Box -->
<!-- Search Results are displayed here -->
{% endblock %}
<!-- Search Unique JS -->
{% block js %}
{{ block.super }}
<script>
$(document).ready(function () {
$("#export").click(function() {
$(this).closest("form").attr('action', "{% query_params_getlist request 'selected_facets' as facets %}{% url 'search_export' facets %}");
});
});
</script>
{% endblock %}
<!-- End Search Unique JS -->
urls.py
urlpatterns = patterns('base.views',
# all the other url patterns go here
url(r'^search_export/(?P<selected_facets>\S+)/$', 'search_export', name='search_export'),
)
base_tags.py
#register.assignment_tag
def query_params_getlist(request, param):
params = request.GET.getlist(param)
if len(params) > 0:
query_string = ""
for p in params:
query_string += p + '&'
return query_string
return 'None'
views.py
def search_export(request, selected_facets):
if request.method == 'GET':
form = AdvModelSearchForm(request.GET)
if form.is_valid():
qs = form.search()
#deal with facets
facets = selected_facets.split("&")
for facet in facets:
if ":" not in facet:
continue
field, value = facet.split(":", 1)
if value:
# faceted fields are stored in a hierarchy, so I check for any document with the given facet or its children
control_value = ControlField.objects.filter(pk=qs.query.clean(value))
if control_value:
value_tree = control_value[0].get_descendants(include_self=True)
sq = SQ()
for index, node in enumerate(value_tree):
kwargs = {str("%s" % (field)) : str("%s" % (node.id))}
if index == 0:
sq = SQ(**kwargs)
else:
sq = sq | SQ(**kwargs)
qs = qs.filter(sq)
response = HttpResponse(content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename="search_results.csv"'
writer = csv.writer(response)
titles = []
rows = []
for result in qs:
row = []
row_dict = {}
properties = result.text #IMPT - this field must be a MultiValueField in the Haystack search_indexes.py file or this won't return a list
for each_prop in properties:
prop_pair = each_prop.split(':', 1)
if len(prop_pair) < 2:
continue
prop_name = smart_str(prop_pair[0].strip())
prop_value = smart_str(prop_pair[1].strip())
if not (prop_name in titles):
column_index = len(titles)
titles.append(prop_name)
else:
column_index = titles.index(prop_name)
if column_index in row_dict:
prop_value = row_dict[column_index] + '; ' + prop_value
row_dict[column_index] = prop_value
for i in range(len(titles)):
if i in row_dict:
row.append(row_dict[i])
else:
row.append('')
rows.append(row)
writer.writerow(titles)
for each_row in rows:
writer.writerow(each_row)
return response
return HttpResponseRedirect('/failed_export/')
Related
I need to initialized a ModelChoiceField value to an instance retrieved in get_context_data() but it does not seem to work. I have tried .initial= but no luck so far.
This is the form
class PostEditForm(forms.ModelForm):
post_category = forms.ModelChoiceField(label='Category',queryset=Category.objects.all(),empty_label="--Select Category--",required=False)
class Meta:
model=Post
fields=('title','summary','content')
and this is the view
class EditPostView(LoginRequiredMixin, AjaxableResponseMixin, UpdateView):
model = Post
form_class = PostEditForm
template_name = 'postapp/editpost.html'
success_url = reverse_lazy('postapp:draftposts')
def get_context_data(self):
context = super().get_context_data()
post=self.get_object()
print(post)
try:
post_category = PostCategory.objects.get(post=post)
print(post_category)
except:
print("Post Category not yet assigned")
else:
context['form']['post_category'].initial = post_category
print(context['form']['post_category'])
return context
and here is the post category model
class PostCategory(models.Model):
post = models.ForeignKey(Post,on_delete=models.CASCADE)
category = models.ForeignKey(Category,on_delete=models.CASCADE)
def __str__(self):
return self.category.title
UPDATE
I forgot to mention I was rendering the form manually, here is the HTML
<select class="form-select" name={{form.post_category.html_name}} id='{{form.post_category.auto_id}}'
{% if form.post_category.field.required %}required{% endif %}>
{% for value, text in form.post_category.field.choices %}
<option
{% if form.post_category.value == value|stringformat:"s" %}selected="selected"{% endif %} value={{value}}>{{text}}
</option>
{% endfor %}
</select>
The value in the initial needs to be PK.
So:
context['form']['post_category'].initial = post_category.id
Also, since post and post category are mapped 1-to-many, PostCategory.objects.get() can return multiple post categories which raises MultipleObjectsReturned error. I suggest using a combination of filter and first so you get the first category if there are many. You will probably change this logic to something that makes sense but it's important to be aware of it.
post_category = PostCategory.objects.filter(post=post).first()
Bonus points: override the get_initial method as it's better practice when using class based views.
def get_initial(self):
initial = super().get_initial()
post=self.object
print(post)
post_category = PostCategory.objects.filter(post=post).first()
if post_category is not None:
print(post_category)
initial.update({'post_category': post_category.id})
print(initial['post_category'])
else:
print("Post Category not yet assigned")
return initial
I am currently learning to code with Angular.js and I am doing a project where I load informations with Ajax queries.
To make it short, I have three levels of datas
-- Skills (competences)
--- Indicators (indicateurs)
---- Results (resultats)
I have a first request that tracks Skills and Indicator for the current skill.
HTML
<div ng-controller="AdminController" data-ng-init="listAllCompetences()">
<!-- Level 1 -->
<div data-ng-repeat="competence in competences" class="content level1 topcompetence" id="competence_{{competence.Competence.id}}">
<div class="level1_title_box">
<h3 class="h3 titre_competence" ng-hide="editorEnabled">{{$index + 1}} - {{competence.Competence.description}} </h3>
</div>
<p><b>Indicateurs de performances : </b></p>
<!-- Level 2 -->
<div data-ng-repeat="indicateur in competence.Indicateur" class="content level2" id="indicateur_{{indicateur.id}}">
<div class="level2_title_box">
<h4>{{$index + 1}} - {{indicateur.description}}</h4>
</div>
<p><b>Results : </b></p>
<p><a ng-click="listAllRestulatsByIndicateurId(indicateur.id)" href="javascript:void(0);">Click here to show results</a></p>
<!-- Level 3 -->
Level 3 shoudl be displayed there...
<!-- End Level 3 -->
<pre>{{resultatsAttendusCallback}} {{resultatsAttendusCallback.length}}</pre>
</div>
<!-- End Level 2 -->
</div>
<!-- End Level 1-->
</div>
When I click on listAllRestulatsByIndicateurId(indicateur.id); function, there is an Ajax request that get all the results for the given indicator.
The point where I have not figured it out yet is how am I supposed to know where to output this since I can have alot of Indicators.
My Angular function
$scope.listAllRestulatsByIndicateurId = function(indicateurID) {
console.log(indicateurID);
var req_resultats_by_indicateur = {
method: 'POST',
url: '../api/resultatAttendus/listByIndicateur',
headers: {'Content-Type': 'application/x-www-form-urlencoded'},
data: {
indicateur_id: indicateurID
}
}
console.log(req_resultats_by_indicateur);
$http(req_resultats_by_indicateur).then(function successCallback(callback) {
if(callback.data.status == 'success') {
$scope.resultatsAttendusCallback = callback.data.data;
console.log(callback);
}
if(callback.data.status == 'error') {
console.log(callback)
}
});
}
Note : The use of callback, may be bad as I named it. It is the returned array from my ajax call.
This line $scope.resultatsAttendusCallback = callback.data.data; give me the array of results with the indicator ID as parent.
But when I execute this function, all my {{resultatsAttendusCallback}} or Results spaces are writing the array. I just want to get the result to be printed in the indicator ID I clicked.
Is there any way to attribute any sort of ID or Class name to those element so I could know on wich element to work and get the angular callback working?
You can pass the current indicateur to the function (The object itself, not its ID) and attach the data directly to that object, then in the view, you'll need to display the returned data that was attached to the object by the ajax request:
<!-- Level 1 -->
<div data-ng-repeat="competence in competences" class="content level1 topcompetence" id="competence_{{competence.Competence.id}}">
<div class="level1_title_box">
<h3 class="h3 titre_competence" ng-hide="editorEnabled">{{$index + 1}} - {{competence.Competence.description}} </h3>
</div>
<p><b>Indicateurs de performances : </b></p>
<!-- Level 2 -->
<div data-ng-repeat="indicateur in competence.Indicateur" class="content level2" id="indicateur_{{indicateur.id}}">
<div class="level2_title_box">
<h4>{{$index + 1}} - {{indicateur.description}}</h4>
</div>
<p><b>Results : </b></p>
<p><a ng-click="listAllRestulatsByIndicateurId(indicateur)" href="javascript:void(0);">Click here to show results</a></p>
<!-- Level 3 -->
Level 3 shoudl be displayed there...
<!-- End Level 3 -->
<pre ng-if="indicateur.resultatsAttendusCallback">{{indicateur.resultatsAttendusCallback}} {{indicateur.resultatsAttendusCallback.length}}</pre>
</div>
<!-- End Level 2 -->
</div>
<!-- End Level 1-->
JS:
$scope.listAllRestulatsByIndicateurId = function(indicateur) {
console.log(indicateur.id);
var req_resultats_by_indicateur = {
method: 'POST',
url: '../api/resultatAttendus/listByIndicateur',
headers: {'Content-Type': 'application/x-www-form-urlencoded'},
data: {
indicateur_id: indicateur.id
}
}
console.log(req_resultats_by_indicateur);
$http(req_resultats_by_indicateur).then(function successCallback(callback) {
if(callback.data.status == 'success') {
indicateur.resultatsAttendusCallback = callback.data.data;
console.log(callback);
}
if(callback.data.status == 'error') {
console.log(callback)
}
});
}
I'm not following your question 100% but I think I know what you are tyring to do. Your callback.data.data is coming back as a JSON string, but you want just a subset of it.
Here is what I do
$scope.resultatsAttendusCallback = MyModel.build(callback.data.data);
...........
angular.module('lodgicalWebApp')
.factory('MyModel',['Column',function(Column){
function MyModel(title, columns){
this.title = title;
this.columns = Column.build(columns); //Column is another Model
}
MyModel.build = function(data){
// //console.log("building variables: " + data);
var models= [];
angular.forEach(data, function(v,k){
models.push(new MyModel(v.name, v.cols));
})
return models;
}
return MyModel;
})
This allows me to return complex JSON and map it to a real object in my code. You don't have to do the forEach in the build either, I do this when my JSON returns a list of MyModels rather than a single one. From there you can add your own ID, attributes, do any data transforms etc.
I am using ng-repeat to create a table of data:
<div class="divTable" ng-repeat="expense in exp.expenses | filter:exp.query">
<div>{{expense.amount | ldCurrency : true}}</div>
...
</div>
A couple of the cells that I am creating are being modified through an Angular filter. In the example above, I am changing the integer to a currency. So the original 4 is changed to $4.00. When I filter the entire list with my exp.query, it does not modify the exp.query search term through the ldCurrency.
The means that if I search on $4, it will not find it, because the backing data is 4, even though $4 is on the page.
I know this is confusing, with the two types of filters that I am talking about here.
How can I search on the data that is being shown on the page and not on the backing data?
You have to create you own filter. What you want to do to is a bad idea, because you are melding the view layer and the model layer.
A example of a filter.
The html:
<input ng-model="query" ng-trim="true">
<span>Finding: </span><span>{{ query }}</span>
<div ng-repeat="product in products | productsFilter: query">
<strong>{{ $index }}</strong>
<span>{{ product.name }}</span>
<span>{{ product.price | currency: '$'}}</span>
</div>
The custom filter:
.filter('productsFilter', [function () {
// Private function: It removes the dollar sign.
var clearQuery = function (dirtyQuery) {
var index = dirtyQuery.indexOf('$');
if (index === -1)
return dirtyQuery;
return dirtyQuery.substr(index+1, dirtyQuery.length-index)
};
// The Custom filter
return function (products, query) {
if (query === '') return products;
var newProducts = [];
angular.forEach(products, function (product) {
var cleanQuery = clearQuery(query);
var strProductPrice = '' + product.price;
var index = strProductPrice.indexOf(cleanQuery);
if (index !== -1) {
newProducts.push(product);
}
});
return newProducts;
};
}]);
The key is in the angular.forEach. There I decide if the product will belong to the new filtered collection. Here you can do the match you want.
You can find the complete example in full plucker example and see a lot of filters in the a8m's angular-filter
I have a Blog model which contains 4 entities, subject, blog, time_created and day_created. I query all the values and assign it to an object which pass to a jinja2 template. In the template I iterate over the object and print each different entity which results in the display of each blog post.
The model for the data is:
class Blog(db.Model):
subject = db.StringProperty(required = True, multiline = True)
blog = db.TextProperty(required = True)
time_created = db.DateTimeProperty(auto_now_add = True)
day_created = db.DateProperty(auto_now_add = True)
And I query all the entries to display as individual posts as such:
posts = db.GqlQuery('SELECT * FROM Blog ORDER BY time_created DESC')
and I pass these out to the template
class Mainpage(BaseHandler):
def get(self):
posts = Blog_posts()
logging.info(posts)
if posts:
end_time = time.time() - start_time
logging.info(end_time)
logging.info(start_time)
userLogin = self.checkLogin()
cookieVal = self.read_secure_cookie('user-id')
if cookieVal:
u_id = cookieVal.split('|')[0]
usr_instance = Users.get_by_id(int(u_id))
if usr_instance:
name = usr_instance.username
if userLogin == True and name:
self.render("blog.html", posts = posts, userLogin = userLogin, user_name = name, end_time = int(end_time))
logging.info("Logged in = " + str(userLogin))
else:
self.render("blog.html", posts = posts, userLogin = userLogin, user_name = "", end_time = int(end_time))
else:
self.render("blog.html", posts = posts, userLogin = userLogin, user_name = "", end_time = int(end_time))
the function render() is :
def render(self, template, **kw):
self.response.out.write(render_str(template, **kw))
I have a main template blogbase.html which I inherit to all other pages and it contains these lines which I use for template inheritance :
<div class="content">
{% block content %}
{% endblock %}
</div>
and in the html file that displays the main blog page and inherits this is :
{% extends "blogbase.html" %}
{% block content %}
{% for post in posts %}
<div class="one_post">
<div class="subject_title">
<div class="day">{{post.day_created.strftime("%d %b %Y")}}</div>
<a href="/blog/{{post.key().id()}}" target="_blank">
{{post.subject}}
</a>
</div>
<div class="post">
{{post.blog}}
</div>
</div>
{% endfor %}
<div class="query">
Queried {{end_time}} seconds ago
</div>
{% endblock %}
I added a debugging line to see whether the query works and it returns the true with the following :
<google.appengine.ext.db.GqlQuery object at 0x7f99f03cd050>
But I am not able to iterate over this object as in the templates, regular html works, but the part between the for loop doesn't render.
Is there something wrong in my query ? Or am I using the templating engine wrong ?
2 things I've noticed.
You are using a query in your templates to iterate over it and do what? Print info? You need to get the entities. Either get them when you query or iterate and fetch.
posts = db.GqlQuery('SELECT * FROM Blog ORDER BY time_created DESC').fetch(1000)
Also if you are passing dicts to jinja you might wanny check that you are using {% for post in posts.iteritems() %} but this is not the case here.
I'm building a RSS feed reader where users can add own feeds and categories. I want to store content all of the the feeds in db and run a script that would fetch that content for all of the feed_url present in db periodically (I had working project where each feed is fetched live and it wasn't good idea)
Now after several attempts I got into the following model structure.
class Category(models.Model):
name = models.CharField(unique=False, max_length=64)
user = models.ForeignKey(User)
def __unicode__(self):
return self.name
class Meta:
ordering = ('name',)
class Feed(models.Model):
feed_url = models.URLField(unique=True)
def __unicode__(self):
return self.feed_url
class UserFeed(models.Model):
feed = models.ForeignKey(Feed)
title = models.CharField(max_length=256)
category = models.ForeignKey(Category)
user = models.ForeignKey(User)
def __unicode__(self):
return self.title
class Post(models.Model):
feed = models.ForeignKey(Feed, related_name='feed_posts')
## starting from here, my cronjob / script populates the data (using feedparser lib)
title = models.CharField(max_length=256)
content = models.TextField()
link = models.URLField(max_length=512)
dt_published = models.DateTimeField()
dt_cached = models.DateTimeField(auto_now_add=True)
def __unicode__(self):
return '%s (%s)' % (self.feed, self.title)
class Meta:
ordering = ('-dt_published',)
What I'm looking for now is ability for pull into a view and later into templates, the following entities:
->Category -> Feed -> Posts
In my previous version where Feed and UserFeed fields were stored in one table/model, I had such view:
def category(request, category_id):
user = request.user
page_title = "Category: "
category = get_object_or_404(Category.objects.filter(id=category_id, user=user)).feed_set.all()
category_name = Category.objects.get(id=category_id)
context = {
'page_title': page_title,
'category': category,
'category_name': category_name,
}
return expand_context_and_render(request, context, 'reader/category.html')
and this is my template:
{% for feed in category %}
<p>{{ feed.title }}</p>
{% for post in feed.post_set.all|slice:"6" %}
<p>{{ post.title }}</p>
<p>{{ post.content }}</p>
<p>{{ post.dt_published|timesince }} ago.</p>
{% endfor %}
{% endfor %}
And it was working as intended.
Now, since I don't want to have multiple copies of feed_url in my database, I think the new models are OK, but I simply can't figure out how to query and display this new structure which has upstream and downstream relationships. I was looking at select_related but didn't get it to work. Could you please help ?
I eventually found the solution myself. It's bit hacky :( but does the job.
def category(request, user_category_id):
user = request.user
page_title = "Category: "
user_category = get_object_or_404(Category.objects.filter(id=user_category_id, user=user))
user_feeds = UserFeed.objects.filter(category=user_category, user=user)
context = {
'page_title': page_title,
'user_category': user_category,
'user_feeds': user_feeds,
}
return expand_context_and_render(request, context, 'reader/category.html')
and template:
{% for user_feed in user_feeds %}
<p><strong>{{ user_feed }}</strong></p>
{% for post in user_feed.feed.post_set.all|slice:"6" %}
<div class="col-md-4">
<p>{{ post.title|truncatechars:"45" }}</p>
<p>{{ post.content|striptags|safe|truncatechars:"85" }}</p>
<p class="feed_date">{{ post.dt_published|timesince }} ago.</p>
<hr />
</div>
{% endfor %}
{% endfor %}