It has been almost three years since there is feedback to implement a Hard-Delete policy for Azure Search indexers. https://feedback.azure.com/forums/263029-azure-search/suggestions/33939013-adding-hard-delete-policy-to-all-of-indexer Has there been any development to this? I am sure this will be requested by a lot of users. If we can't have this feature anytime soon and if we can't implement Soft-delete in our applications, are there any alternatives/ideas we can try?
There's currently no hard delete policy for indexers. For blob indexers you may be able to use the native blob soft delete policy, which requires enabling "Blob soft delete" on the storage account so you don't have to manage the soft delete metadata.
For other data source types, one alternative is to use the REST API to directly remove documents from the index when they get removed from the data source, and synchronize them externally.
Related
Interested to know if anyone has successfully implemented Superset as there customer analytics platform?
We are currently evaluating this, however, some struggle with restricting access to subsets of a dataset/source, or limit access to the specific row data.
For example companyB should only see data relevant to companyB. This is of course a mandatory requirement.
There are a lot of similar questions raised, so would be keen to know if someone has successfully accomplished this.
We are in the process of customizing superset to do something similar (without forking), though it doesn't involve row-level granularity. I was able to accomplish view-level access with a combination of FAB permissions and a custom security manager. Basically:
User logs into superset through OAuth2 with our API.
Security manager makes request to our API for a list of things the user can access.
Superset builds datasources only for those items.
For this to work, we created a custom role that forbids users from doing things like creating new databases/tables, and add each user to that role once they log in. At this point, you may want to make a connector for your own purposes so users can refresh datasources when they want/need to. Superset exposes a config setting for custom connectors so you don't have to modify source to load it. The Druid connector is a good example of this.
So, to answer your question: sort of. Table-level/view-level access control is definitely doable, though it will be a bit of effort. Row-level access control? Probably not, unless your database engine of choice supports row-level access control.
I was wondering how so many job sites have so many job offers/information regarding other companies' offers. For instance, if I were to start my own job searching engine, how would I be able to get the information that sites like indeed.com have in my own databases? One site (jobmaps.us) says that it's "powered by indeed" and seems to be following the same format as indeed.com (as do all other job searching websites). Is there some universal job searching template that I can use?
Thanks in advance.
Some services offer an API which allows you to "federate" searches (relay them to multiple data sources, then gather all the results together for display in one place). Alternatively, some offer a mechanism that would allow you to download/retrieve data, so you can load it into your own search index.
The latter approach is usually faster and gives you total control but requires you to maintain a search index and track when data items are updated/added/deleted on remote systems. That's not always trivial.
In either case, some APIs will be open/free and some will require registration and/or a license. Most will have rate limits. It's all down to whoever owns the data.
It's possible to emulate a user browsing a website, sending HTTP requests and analysing the response from a web server. By knowing the structure of the HTML, it's possible to extract ("scrape") the information you need.
This approach is often against site policies and is likely to get you blocked. If you do go for this approach, ensure that you respect any robots.txt policies to avoid being blacklisted.
I am developing a mobile ticketing system, and I'm reviewing my requirements against embedded SymmetricDS. The only sticking point so far is that I can't find any information directly addressing the question of row-level security.
Use Case:
Some of the mobile point-of-sale nodes will be logged in as sellers, some as managers. Everybody can view everybody else's sales. Sellers can create new sales, but never modify them. Managers can modify existing sales, but not delete them.
Problem:
I don't have strict control over the Android mobile POS units, so they aren't trustworthy -- it's not realistic to prevent a malicious seller from decompiling the APK and creating a malicious client node. My environment has security requirements, such that a malicious seller cannot be allowed to modify the sales table on the server. I can trust that the hypothetical malicious seller does not have access to manager credentials, and I can trust that the server software is secure.
Questions:
Is server-side row-level security a job for a load filter?
Can the filter script get access to the node_id originating the change?
Can the filter script get access to the authentication credentials used to register the originating node?
Yes. Some of its is possible implementing with a writer filter, like the update limiting to a certain group of nodes. Disabling deleting a row from a node just means that delete trigger shouldn't be created at all for that node group
Yes, it's possible. Methods that are implemented in an extension of an abstract load file accept arguments that carry external ID of the caller (org.jumpmind.symmetric.io.data.Batch#getSourceNodeId and DataContext#getBatch())
Basic authentication is the same for all nodes. There's a password that is automatically generated when handshake is performed and it's stored in the db. Let your load file implement symmetricds interface ISymmetricEngineAware which allows for injection of an engine that can be used to access the database
I am implementing an Azure Mobile Service and I have a set of objects that can be accessed by multiple users, potentially at the same time. My problem is that I can't really find a lot of information on how to handle the potential concurrency issues that might result from this.
I know "Azure Tables" implements an E-Tag check. Is there an out-of-the-box solution for Azure Mobile Services for this one with SQL ? If not, what is the approach I should be going for here.
Should I just implement the E-Tag check by hand? I could include a GUID of the object that is generated every time the object is saved and checked when saving. This should be relatively safe way to do it?
For conflicts between multiple clients, you would need to add a detection/resolution mechanism. You could use a "timestamp" (typically a sequentially incremented version number) in your table schema and check it in your update script. You could fail an update if the timestamp used by a client for reading is older than the current timestamp.
If you want to use ETags via http headers, you could use Custom APIs. We are looking into enabling CRUD scripts to also set headers but that is not available today. Separately, we are looking into the offline scenarios as well.
(Prog Manager, Windows Azure Mobile Services)
I'd like to have a single instance of Solr, protected by some sort of authentication, that operated against different indexes based on the credentials used for that authentication. The type of authentication is flexible, although I'd prefer to work with open standards (existing or emerging), if possible.
The core problem I'm attempting to solve is that different users of the application (potentially) have access to different data stored in it, and a user should not be able to search over inaccessible data. Building an index for each user seems the easiest way to guarantee that one user doesn't see forbidden data. Is there, perhaps, an easier way? One that would obviate the need for Solr to have a way to map users to indexes?
Thanks.
The Solr guys have a pretty exhaustive overview of what is possible, see http://wiki.apache.org/solr/MultipleIndexes