I've created a gadget for our CRM consultants that allows them to present data from an oData source in CRM. At the moment, it will connect to any data source but for customer sites we need to develop an oData service using WCF each time for each data source.
Does anyone know if there's a decent generic tool out there that can retrieve data from SQL Server, present it (via IIS) as oData and that can be configured without Visual Studio by a non-developer?
We (the WCF Data Services team) have heard this ask a couple of times; what follows are a few of my thoughts in no particular order.
We haven't heard the ask a lot. There's a reasonable amount of work to do here, and without sufficient asks it's hard to justify. That said, there's nothing stopping the community from spinning up an effort to achieve this (hint, hint :)).
There's a number of questions you would need to answer. For instance, what sort of default limitations would the provider have? Would you really want to allow arbitrary expands on something that's probably a production database server? What about permissions? What about read/write?
What happens for mutable schemas? Is this a completely dynamic provider? How much overhead is there in scanning the database schema, and how frequently would the database schema need to be scanned?
How would clients take advantage of a dynamic OData service? Most clients use some form of code generation to make interacting with the service easier.
These thoughts aren't really intended to dissuade at all, but hopefully they give you some things to think about should you attempt to create a generic provider on your own. If you do so, I'd love to hear about it.
Related
Hey there fellow Stackoverflowers,
In our company we have several application stacks running on different types of databases (MySQL, PostgreSQL, MS SQL, Azure SQL,..). For monitoring purposes we use some scripted queries on the databases of all these application stacks, with Nagios reporting back the results in an email.
Now, since our support team would also like easy access to these queries in order to manually run them or modify them, we were considering building an application specifically designed to be able to store, run and modify queries that can be executed on any of the above listed database types and offering both a user-friendly webinterface and a REST API with JSON output for our new reporting stack based on SENSU, to be deployed in a few months.
My personal belief is that a tool like this must already be out there, since the use case for it is so generic. However, googling did not yield any results even closely resembling what I am looking for.
So my question to you is: Do you know of such a tool? If you had to build it yourself: what would your approach be? We're mostly a Java/C++ team, but are open to all options.
Some or may be all of this stuff can be done by an existing API called NAGIRA. Look it up on Google. This will definitely give you all the results in JSON format. Also i think it would allow you to run checks manually. So you can may be build a little front end and call this API to achieve what you want.
A little late of a reply, but check out http://cloudmonix.com -- it offers ability to create metrics based on custom SQL queries, supports SQL Azure, SQL Server, MySQL, and Oracle. Also integrates with Nagios (and Zabbix)
I am looking for the easiest method to pull data from a WCF and load it into a SQL Server database.
I have a request for some data in our database. To learn EF I created a Entity Model based on a view that pulls the data (v_report_all_clients). I created the WCF service based on that model and tested it with a simple WPF datagrid. The WCF service exports a collection of v_report_all_clients objects which easily load into the simple data grid.
The user requesting the data is going to load the results into their database. I believe they have access to SSIS packages but I'm not sure using SSIS is the easiest method. I tried two approaches and neither were successful let alone easy. Briefly the methods I tried are:
Use Web Service Task - This works but unless I missed something you have to send the output to an XML file which seems like a wasteful middle step.
Use a Script Component - I am running into various issues trying to get this to work even after I follow some online examples. I can try to fight through the issues is this ends up being the best method but I am still not sure if this is the easiest. Plus the online examples didn't include the next logical step of loading into a database.
This is my first attempt to use a WCF as a means to distribute data from our database to various users. I'd like to leverage this technology for some of our larger web based reports that end up being almost an entire table dump. But if the users cannot easily integrate the WCF output into something they can use then I might have stick with web based reporting.
WCF services generally do not "contain" any data. From your description I am assuming the data you want to pull is probably contained in a SQL database somewhere on the lan/wan.
If it's in a database, and you want to put it in another database, then there is already a technology for doing that, it's called SSIS, which you're already using.
So my solution would be: don't use WCF for this task.
If you're going outside the local lan/wan for the data, then there are other ways to replicate the data locally.
Appreciate this does not answer your question as asked.
EDIT
Uncertain how database ownership "chaining" works - but suffice to say that either of the two methods you're proposing above (built in task or script task) both would appear to be eminently feasible. I am not a SSIS guru so would struggle to make a recommendation as to which one to use.
However by introducing a wcf service call into a ETL process you need to bear a few things in mind:
You are introducing an out-of-process call with potentially large performance implications. WCF is not a lightweight stack - opening a channel takes seconds rather than the milliseconds it takes to open a DB connection.
WCF service calls fail all the time. Handling timeouts and errors on the service side will need to be planned into your ETL process.
If you want to "leverage the service for other uses", you may need to put quite a lot of upfront time into making sure the service contract is useful, coherent, and extensible - this is time you would not have to spend going direct to the data.
Hope this has helped you.
I think the idea given by preet sangha is the best answer. I've experimented with a few possible solutions and I think a WCF Data Service is the easiest solution. After installing the OData add-on package for SSIS I was able to pipe the WCF Data Service directly to another table (or any other SSIS destination). It was very simple and easy in my opinion.
I think a second place vote might go to Web API which will give you more flexibility and control. The problem is that comes at a price of requiring more effort. I am also finding some quirks in Web API scaffolding.
I am using MonoDevelop and MonoTouch to create an application which needs to access SQL Server 2008 tables. I understand that if you use objective-C then the only way to access SQL Server is via web service but what about MonoTouch. Is there anything better than coding a web service when using MonoTouch?
Better? I would say no there isn't. Creating a Web Service is the way to go in this scenario. You keep your data access layer on the server, you keep your security footprint small, and you enforce data validation on the server-side. All of this keeps your MonoTouch application smaller and focused on the Presentation layer and Business Logic Layer.
From your question, I am assuming you would rather stay away from WCF but it really is the way to go. This way you set yourself up to work with MonoDroid, a website, Windows Phone 7, or other platforms without additional Data Access (and possibly Business Logic) layers needing to be written. Even if my research didn't suggest that there isn't a SQL Server connector for MonoTouch, I would still recommend going with WCF. From what I have seen though, it doesn't seem like you have a choice here.
I recently attended the Microsoft Mix event and sat through these OData presentations. It includes code samples for obtaining data via objective C or any other subscriber/device/language from an OData provider (including SQL Server). They are well worth the watch if you're interested in data access technologies.
http://channel9.msdn.com/Events/MIX/MIX11/FRM10
http://channel9.msdn.com/Events/MIX/MIX11/FRM11
http://channel9.msdn.com/Events/MIX/MIX11/FRM16
I've worked with many APIs and it's never usually an easy task. Messing about with POST requests and then trying to handle the XML is a pain. And I thought wouldn't it be easier for both user and developer if they could just directly interact with the database.
Is it possible to create a user which API users would connect to then assign that certain privilages? For example they would only be able to select from particular tables and columns. And basically make it so they can't do anything malicious or anything you don't want.
I realise that there is a lot more than just taking data so there would be certain limitations there however selecting is probably what goes on the most when it comes to API usage.
Is this a practical idea? Is it secure? I'm really not sure, I'm the furthest thing from a professional here, it's just an idea.
You could set up a RESTful API that can speak directly to a mySQL database like PHPRestSQL. It can do all the dirty work for you, but you would have full freedom in implementing new functions or restrictions.
What do you mean exactly by API, which API are you talking about?
This sounds more like a design decision. If I understand correctly, you want to interact with the User layer and Database / Persistence layer of an application. In general this is a bad idea. First it really reduces code reuse. This may not be a concern at your point in the development but it is a good idea to learn best practices. The layers I usually follow are:
Model-View-Controller
Service
Persistence
Model / Domain
You can see here that MVC (user interface) is separated from the model by at least two layers. This is usually more secure, and promotes code reuse.
Yes you can do this with any client / server database system (if it is a database server there must be a way to connect to it.)
It is not done much because of a number of issues.
Maintenance is hard
Security is worse
In general there is no benefit.
Basically it causes headaches and does not really provide anything which is good.
The two most important counter-questions are:
1) Is the underlying DB already determined, or can you choose one?
2) What sort of DB operations do your users really need to perform? If "select" is really enough then yes, it probably does make sense to expose the data via a "read only" web service. But if you want to update, delete, make stored procedure calls, etc. then you're going to need something like SQL and it's way hard to build a web services API for that.
If the answer to counter-question 1 is "I can choose", then take a look at CouchDB, which already has a RESTful API (http://wiki.apache.org/couchdb/HTTP_REST_AP) built for it.
Yes, almost all databases allow you to create users with only select access to a specific schema. I've used this to give advanced Excel users ODBC access without worrying that they will mess anything up. Use very sparingly--it has always created maintenance difficulty because people end up using parts of your schema in ways that you didn't intend (or had plans to replace).
You can connect Access to any database - Oracle, for example.
However, it's not necessarily a good idea - for security and data integrity reasons.
Is it possible to access the database backend of a sharepoint server? My company uses Sharepoint to store data and pictures of various assets. Ideally I would be able to access the data and display it in my application to allow users both methods of access.
Before I go talk to the IT department I would like to find out if this is even possible?
Edit: From rails on linux? (Yes, I know I'm crazy)
Agree with Adam. Querying the Sharepoint Database is a big no-no, as Microsoft does not guarantee that the Schema is in any way stable. Only access the database if there is really no other way.
As for Sharepoint, usually the Lists.asmx Web Service is what you want to look at first.
http://www.c-sharpcorner.com/UploadFile/mahesh/WSSInNet01302007093018AM/WSSInNet.aspx
http://geekswithblogs.net/mcassell/archive/2007/08/22/Accessing-Sharepoint-Data-through-Web-Services.aspx
yikes! :)
look at the web service and .net API before going direct to the database. i've used both and they provide plenty of flexibility (including building your own web services on top of the API if necessary). API for on server clients, web services for off server clients.
Just a small comment. Never ever go to the database direct. If there is no way to do it via published and supported API's, then there is no way to do it. End of story. This applies even to when you are "just reading data", as this can still cause significant issues.
Just to support the above if you ever take a look at the SQL tables that sit behind SharePoint you'll realise why its not recommnded or supported to access the database direct. MADNESS!