I'm considering Talend's Open Studio for a Data Integration / ETL project, and I can't seem to find a list of formats for which it can input from or output to out of the box. For instance, I'm comparing it against Pentaho's Kettle, which I found supports File System I/O, MS Excel, Access, XML, JSON, SAP, various SQL databases, etc., out of the box. Where can I find similar information for Talend?
By "out of the box" I mean without having to code some Java plugin to do the conversion manually.
One way to look into this is to check out the Talend components reference guide from their download page: http://www.talend.com/download/data-integration (click on the User Manuals tab).
The Reference Guide PDF describes all components that come with Talend out of the box and will include all possible formats you can use.
Here is a list of all the Talend connectors, including the connection methods:
http://www.talendforge.org/components/
All of the formats that you mentioned are supported.
Related
I want to use Tableau integration with SQL Server for my business reporting (generate Excel and PDF files). However I do not want to use the Tableau dashboard as the user interface for collecting input data. I already built our web applications to do that.
So the requirements are:
Custom web applications (no Tableau Dashboard)
Tableau integration with SQL server
Output Excel/PDF
Is it possible to use a custom web application to replace the Tableau Dashboard for collecting user input?
Can anyone share any documentation?
If what you want is a custom web app with some visualizations implemented by Tableau Server, yet to have those visualizations embedded and controlled by your custom application, then Tableau has a lot of options for you.
See the sections on "Embedding" visualizations in the Tableau help pages. You may also want to use the Javascript API (start with the excellent 5 minute tutorial). That API allows you to control interaction between your application and the embedded visualization to make the integration more seamless, and customized to your needs.
To avoid forcing your users to authenticate with Tableau Server, read about Tableau's solution to Single Sign On and/or read about Trusted Authentication (which lets your application vouch for a user's identity if you have a core based license).
All these features are described in the Desktop, Server or Developer guides on Tableau's help pages. I recommend embedding using the object HTML tag instead of the iframe tag. Both work and both are documented, but the object tag gives you more control, works well with the JavaScript API and avoids some of the legacy HTML issues surrounding iframes.
If instead, as you stated, your only goal is to produce Excel or PDF output, Tableau can do that - one easy way is to append .pdf or .csv to URLs. Again read the documentation on embedding. But if that is the only goal, especially for CSV, Tableau is an expensive way to simply generate a CSV file. I'd give some thought to your requirements. There are reporting packages that are focused on hard copy tabular reports (sometimes called report factories) - that's not Tableau's primary focus.
One approach that can be helpful, use a modest investment in Tableau Desktop to quickly prototype your visualizations and reports. Verify you are producing the right outputs and that you have the right data to create them. Then you can decide whether Tableau is the best way to meet your requirements in a production environment. Even if you use a different approach in production, it can be very helpful for speeding that initial investigation.
Earlier we have used Microsoft OLEDB JetProvider in SSIS package. After recent update from Microsoft, now we are facing issues with SSIS package. So we have decide to export data to excel using open XML. What should be the best approach for implementation since still we are using (xls) version 1997-2003.
Note: We already tried Microsoft Access Database engine 2010 Redistributable.
from my point of view, you have the following options (all about the ScriptTask unfortunately):
Call REST API and create a document there (using Open XML SDK). It's easy to develop, support and deploy
Use Open XML SDK directly in the ScriptTask
I would recommend following the first approach, but it all depends on your system though
UPDATE:
Following the first option, you have to develop a small Web API Service. Here is the link with an example on C#
Per the second option, in order to use external DDLs, such as OpenXML, you have to register it in the GAC (if the installer doesn't). Here is the link with an example of using external libraries.
If you are going to follow this option, I would recommend you develop a DDL that would work with Open XML directly and have simple API for calling it from SSIS Script Task. You will register your DDL in GAC and have a link in Script task. It will help you avoid a number of debugging issues.
Serilog is a logging library that has Sinks (adapters) to many log viewing/analysis services (like Glimpse or Loggly). That is cool but all of them require browser to see the logs.
Is there a ready GUI control for desktop applications (using winforms, WPF, etc..) for log entries that works with serilog?
P.S. We can take IObservable serilog Sink and create our own "LogGrid" but we would prefer to not to reinvent a bicycle.
It depends on where you log the messages from Serilog. You can log to numerous "sinks" so most GUI tools will depend on what sink you are using. The list of provided sinks for Serilog is here: https://github.com/serilog/serilog/wiki/Provided-Sinks
Some options include:
Log Parser
If you are using some file-based sinks, then you can use the Microsoft Log Parser available here: https://www.microsoft.com/en-us/download/details.aspx?id=24659
From the site:
Log Parser 2.2 is a powerful, versatile tool that provides universal
query access to text-based data such as log files, XML files and CSV
files, as well as key data sources on the Windows operating system
such as the Event Log, the Registry, the file system, and Active
Directory.
Log Parser Lizard GUI
If you want a shiny UI to search the log files using log parser, check out Log Parser Lizard GUI here: http://lizard-labs.com/log_parser_lizard.aspx
From the site:
Log Parser Lizard is a GUI for Microsoft Logparser, definitely the
best one available on the market today. Log Parser is a very powerful
and versatile query software tool that provides universal query access
(using SQL) to text-based data, such as log files, XML files, and
TSV/CSV text files, as well as key data sources on the Microsoft
Windows operating system, such as the Windows Event Log, IIS log, the
registry, the File System, the Active Directory services and much
more.
Seq
This does NOT meet your "non-browser based UI" requirement, however Seq does a great job of providing a web-based interface to search and analyze logs with built-in support for Serilog. It is available here: https://getseq.net/
From the site:
Seq is the easiest way for .NET developers to capture, search and
integrate structured log events. Compared with traditional plain-text
logs, structured logs cut through the complexity of distributed and
asynchronous applications.
Have a look at https://github.com/RolandPheasant/TailBlazer to see how Roland did it.
I started working with PostGIS in Action. The book shows a bunch of neat graphics created from various queries, but there is no mention of how to get these graphics to show up. The queries are working fine.
All of the tutorials I have found online appear outdated, suggesting, for example, to install plugins that are no longer available. I know I'm missing something obvious, but I have no idea what it would be.
I'm using Windows 7, PostgreSQL 9.3, PostGIS 2.1.
UPDATE
Thanks a lot guys. I settled on QGIS and it works as promised. So hard to pick either answer as they are both brilliant and informative.
There is no built-in GUI for Postgis queries. You have a number of options:
Connect to Postgis directly from QGIS or other GIS package that supports direct connections.
Use ogr2ogr utility to convert to another format such as shp or kml and use ArcGIS, Google Earth, etc to visualize the results.
Write the results out as WKT (well-known text) or GeoJSON and load them into a web mapping library such as OpenLayers. There is an example of a WKT reader here: OpenLayers WKT example. GeoJSON is a format that has gained a lot of traction, as it is very amenable to manipulation in Javascript and hence mapping libraries such as OpenLayers or Leaflet.
If you decide to use the ogr2ogr route, here is a cheat sheet provided by the authors of the Postgis book: http://www.bostongis.com/PrinterFriendly.aspx?content_name=ogr_cheatsheet
I am sure there are other options, but these are the ones that sprung to mind.
PostGIS has no GUI, it's a database extension and so it's purely backend.
If you want graphicals you need a 3rd party gis application to do that. Some worth trying might be:
QGIS - probably the most popular FLOSS gis application
OpenJUMP - a much simpler although quite limited FLOSS
ArcGIS - most advanced and most expensive
Your book probably has a note explaining where the screenshots come from, try searching towards the back of the book.
I am searching for the proper way to build an online database application to behave similar to an Excel datasheets that I have.
I have a working excel file with many calculations, and I need to port this to my WordPress website as a service.
My clients would be able to log into the web app section in WordPress, and manipulate their own data sheet, and get results from all the calculations.
Any idea or direction regarding recommended platform or programming language that can do that ?
Sincerely
Dan
If you want to create a web calculators based on an Excel file I would recommend you to take a look at “Appizy”. It converts instantly a spreadsheet into a standalone web calculation tool (HTML+CSS+JAVASCRIPT). You don’t need to hardcode the whole interface and calculation formulas!
As your Excel file is converted in HTML you can download it and paste it in your WP installation.
If you want to include database you might need to had a bit of PHP on top...
Let me know if it helped you!