I've been struggling with implementing this. I'm trying to build a dynamic queryset where the conditions will be based on user entry. A user can decide to search by first or last name or a combination of both. If searching by only last name the firstname query will not be added to the queryset. I currently have it working for a search with all the fields entered.
results = documents.objects.filter(
Q(f_filer__filer_first_name__istartswith=request.GET.get('first_name',
'')) & (f_filer__filer_last_name__istartswith=request.GET.get('last_name',
'')) &
Q(f_office__o_office_name__istartswith=request.GET.get('office_name', ''))
& Q(f_doc_year__exact=request.GET.get('report_year', ''))
& Q(f_report_id__exact=request.GET.get('filing_type', ''))
).values('f_filer__filer_first_name',
'f_filer__filer_last_name',
'f_office__o_office_name',
'f_date_received',
'f_start_travel_date',
'f_end_travel_date',
'f_doc_year',
'f_report__r_title')
You can just concatenate filter like this:
queryset = documents.objects.all()
first_name = request.GET.get('first_name')
last_name = request.GET.get('last_name')
if first_name:
queryset = queryset.filter(first_name=first_name)
if last_name:
queryset = queryset.filter(last_name=lastname)
I have shorten the filter arguments for simplifying the example
You can join Q objects using |, which acts as an "or":
first_name = request.GET.get('first_name', '')
last_name = request.GET.get('last_name', '')
Q(
f_filer__filer_first_name__istartswith=first_name,
f_filer__filer_last_name__istartswith=last_name
) |
Q(f_filer__filer_first_name__istartswith=first_name) |
Q(f_filer__filer_last_name__istartswith=last_name)
If your application is search-heavy, it might be worth looking into setting up a proper search backend.
Do you use Postgres? If so, using Django's Postgres Full text search would probably simplify things and help make the code more maintainable.
If you don't use Postgres, setting up a search backend with django-haystack would give you similar benefits and allow users to construct Google-like queries using AutoQuery. For example, using - to exclude a term and quotes for an exact phrase.
Related
I was able to create a data masking policy for a json column for the top level keys with the following, but couldn't figure out to go to deeper layers in json. Anybody has done that?
CREATE OR REPLACE MASKING POLICY json_mask_test AS
(val variant) returns variant ->
CASE
WHEN invoker_role()='ADMIN' THEN val
ELSE object_insert(
object_insert(
object_insert(val, 'pii_field', '***', true),
'address','***', true),
'lastName','***', true)
END
If object_insert is the only way to create a masking policy on a json field, looks like it's limited to top level keys.
I was using the example for On Variant Data
Also as a side effect this policy inserts the keys into the json fields when the keys don't exist in original field. To be able to eliminate this would be desirable.
Edit:
I have used this json for the example above
{"regular_field": "regular data", "pii_field": "pii data"}
I was trying to mask LastNames in a json like the following
'{"root":[{"employees":[
{"firstName":"John", "lastName":"Doe"},
{"firstName":"Anna", "lastName":"Smith"},
{"firstName":"Peter", "lastName":"Jones"}
]}]}'
In the case above we create views to flatten json fields for consumption.
To meet the requirement, the masking policy can be applied to the view.
View definition :
CREATE OR REPLACE VIEW v_json_mask AS
SELECT n.value:firstName::string firstName, n.value:lastName::string lastName
FROM json_test,
lateral flatten (input => v:root) r,
lateral flatten (input => r.value) e,
lateral flatten (input => e.value) n
I have created another masking policy for last name masking and applied to the view
CREATE OR REPLACE MASKING POLICY lastName_mask AS
(lastName text) returns text ->
CASE
WHEN invoker_role()='ADMIN' THEN lastName
ELSE '*masked*'
END;
ALTER VIEW v_json_mask MODIFY COLUMN lastName SET MASKING POLICY lastName_mask;
Running SELECT * FROM v_json_mask with any role not specified in masking policy returns
FIRSTNAME LASTNAME
John *masked*
Anna *masked*
Peter *masked*
This also works and was the recommended practice for masking data before Snowflake created data masking policies. Just use RBAC to control access to the JSON table and use a Secure View.
CREATE OR REPLACE SECURE VIEW v_json_mask AS
SELECT n.value:firstName::string firstName,
CASE WHEN current_role()='ADMIN' THEN n.value:lastName::string
ELSE '*masked*'
END as lastName
FROM json_test,
lateral flatten (input => v:root) r,
lateral flatten (input => r.value) e,
lateral flatten (input => e.value) n
does Salesforce offer a way to obtain all Objects like Account, Contact etc and populate them in a SQL table with certain columns like
ObjectEntity, FieldName , FieldType ?
I'm pretty sure the only way to achieve this would be by using the Schema.sObjectType and Schema.sObjectField. Here is a link to the documentation for getting all sObjects. You will basically call the Schema.getGlobalDescribe() method which will return you a map of sObjectTypes with their sObject name as the key. Then you'll need to call getDesribe() on each sObjectType get the fields of the object from that Describe result. You'll again need to call getDescribe() on each sObjectField in order to have access to the columns you wanted (eg. FieldType). Here is the documentation on that You could save each DescribeFieldResult into a list that goes into a Map of > with the sObject name as the key, then you could do what you want with them... Put them in a table if you like. Keep in mind this is all going to be very expensive when it comes to CPU time. You may even run into some governor limits.
Here is a little example you can run using Execute Anonymous in the developer console where the sObject Name and all its field names and types are printed to the debug logs
Map<String, sObjectType> objects = Schema.getGlobalDescribe();
for(String objName : objects.keySet()){
system.debug('=======' + objName + '=========\n Fields: ');
sObjectType o = objects.get(objName);
DescribeSobjectResult oDRes = o.getDescribe();
Map<String, Schema.SObjectField> fields = dResult.fields.getMap();
for(Schema.SObjectField f : fields.values()){
DescribeFieldResult fDRes = f.getDescribe();
system.debug('name: ' + fDRes.getName() + ' | type: ' + fDRes.getType());
}
}
Hope this helps
I have a requirement to pull data in a specific format and I'm struggling slightly with the ROWTOCOL function and was hoping a fresh pair of eyes might be able to help.
I'm using 10g Oracle DB (10.2) so LISTAGG which appears to do what I need to achieve is not an option.
I need to aggregate a number of usernames into a string delimited with a '$' but I also need to concatenate another column to to build up email addresses.
select
rowtocol('select username_id from username where user_id = '||s.user_id|| 'order by USERNAME_ID asc','#'||d.domain_name||'$')
from username s, domain d
where s.user_id = d.user_id
(I've simplified the query specific to just this function as the actual query is quite large and all works except for this particular function.)
in the DOMAIN Table I have a number of domains such as 'hotmail.com','gmail.com' etc
I need to concatenate the username, an '#' symbol followed by the domain and all delimited with a '$'
such as ......
joe.bloggs#gmail.com$joeblogs#gmail.com$joe_bloggs#gmail.com
I've battled with this and I've got close but in reverse?!.....
gmail.com$joe.bloggs#gmail.com$joeblogs#gmail.com$joe_bloggs
I've also noticed that if I play around with the delimiter (,'#'||d.domain_name||'$') it has a tendency to drop off the first character as can be seen above the preceding '#' has been dropped from the first email address.
Can anyone offer any suggestions as to how to get this working?
Many Thanks in advance!
Assuming you're using the rowtocol function from OTN, and have tables something like:
create table username (user_id number, username_id varchar2(20));
create table domain (user_id number, domain_name varchar2(20));
insert into username values (1, 'joe.bloggs');
insert into username values (1, 'joebloggs');
insert into username values (1, 'joe_bloggs');
insert into domain values (1, 'gmail.com');
Then your original query gets three rows back:
gmail.com$joe.bloggs
gmail.com$joe_bloggs#gmail.com$joebloggs
gmail.com$joe_bloggs#gmail.com$joebloggs
You're passing the data from each of your user IDs to a separate call to rowtocol, which isn't really what you want. You can get the result I think you're after by reversing it; pass the main query that joins the two tables as the select argument to the function, and have that passed query do the username/domain concatenation - that is a separate step to the string aggregation:
select
rowtocol('select s.username_id || ''#'' || d.domain_name from username s join domain d on d.user_id = s.user_id', '$')
from dual;
which gets a single result:
joe.bloggs#gmail.com$joe_bloggs#gmail.com$joebloggs#gmail.com
Whether that fits into your larger query, which you haven't shown, is a separate question. You might need to correlate it with the rest of your query.
There are other ways to string aggregation in Oracle, but this function is one way, and you already have it installed. I'd look at alternatives though, such as ThomasG's answer, which make it a bit clearer what's going on I think.
As Alex told you in comments, this ROWTOCOL isn't a standard function so if you don't show its code, there's nothing we can do to fix it.
However you can accomplish what you want in Oracle 10 using the XMLAGG built-in function.
try this :
SELECT
rtrim (xmlagg (xmlelement (e, s.user_id || '#' || d.domain_name || '$')).extract ('//text()'), '$') whatever
FROM username s
INNER JOIN domain d ON s.user_id = d.user_id
I am implementing full text search on Jackrabbit Repository. After going through the examples given at http://jackrabbit.apache.org/ocm-search.html, I am able to write perform full text search on repository when only 'and' is required in the predicate. For example:
select * from test where name like '%abc%' and type = 'mainPage' and language = 'english'
can be written as
Filter filter = queryManager.createFilter(Paragraph.class);
filter.addContains('name', 'abc');
filter.addEqualTo('type', 'mainPage');
filter.addEqualTo('language ', 'english');
But, if I try to write the OCM implementation for the following query
select * from test where (name like '%abc%' or name like '%def%' ) and type = 'mainPage' and language = 'english'
as given bellow, I am getting empty list
Filter mainFilter= queryManager.createFilter(Paragraph.class);
Filter filter = queryManager.createFilter(Paragraph.class);
filter.addContains('name', 'abc');
Filter filter1 = queryManager.createFilter(Paragraph.class);
filter.addContains('name', 'def');
mainFilter = filter .addOrFilter(filter1 );
mainFilter .addEqualTo('type', 'mainPage');
mainFilter .addEqualTo('language ', 'english');
I think I am not able to use OCM full text search properly. Please suggest me the right way to implement OCM full text search in which predicate contains a large number of 'and' and 'or' conditions.
When I used
filter.addLike('name','%def%');
It's working fine. I am still wondering why addContains() is not working.
print("select CustomerNo, CustomerName, Address, City, State, Zip,
Phone, Fax, ContactName, Email
from Customers where CustomerName like '%field%'");
Hi all. This is a simple question but I wasn't able to figure since I'm pretty new to tsql and sql in general.
I use the above stored procedure to do search. My question is for '%field%'. What variable do you use or how does it work in tsql? for example, "where Customers = #CustomerNo". how about for wildcard? how do you pass in a variable along with wildcard? I guess i can do "%" + "field" + "%" in the code but is there a way not to do that?
Wildcards are simply part of a string literal, e.g. '%field%' is just a string.
You can concatenate the wildcards onto your string and then use the string:
#Pattern = '%' + #CustomerName + '%';
...WHERE CustomerName LIKE #Pattern
Or else you can write an expression in the SQL involving concatenation:
WHERE CustomerName LIKE '%' + #CustomerName + '%'
There's no other magic solution for this.
It is very simple. "=" and "Like" are both operators. What you can do after one you can do after the other.
So, if in C# and using SQLClient calls, you can say:
string value;
...
value = "Some name";
...
myCommand.CommandText = "Select...from Customers Where CustomerName Like #Var";
myCommand.Parameters.AddWithValue("#Var", "%" + value + "%");
myCommand.ExecuteNonQuery();
If you use =, you say "equal", which won't use wildcards.
If you use LIKE, which only work on text fields, it can use wildcards.
There is no way you can get wildcard matches with =.
Note that depending on the data, a wildcard search might do a table-scan, so I would make sure that's what you want before you allow it.
For instance, this will do a table-scan:
WHERE CustomerID LIKE '%1'
ie. all customers who have a customer identifier (which is text) that ends with a 1. That can't be solved with an index.
Final thoughts. I'm not 100% sure I understand exactly what you're asking. Could you please clarify. What specifically do you mean by "pass in a variable along with wildcard"?