all.
I am considering for real-time data traffic diameter interface on telecommunications.
I want to use apache camel for the interfaces.
Can camel support this industry?
Mobile service for data require diameter protocol including gy, ro, so and so on.
I can’t judge about this issue.
Please advice to me.
Thank you.
Related
I am using Apache camel for quite long time and found it to be a fantastic solution for all kind of system integration related business need. But couple of years back I came accross the Apache Nifi solution. After some googleing I found that though Nifi can work as ETL tool but it is actually meant for stream processing.
In my opinion, "Which is better" is very bad question to ask as that depend on different things. But it will be nice if somebody can describe more about the basic comparison between the two and also the obvious question, when to use what.
It will help to take decision as per my current requirement, which will be the good option in my context or should I use both of them together.
The biggest and most obvious distinction is that NiFi is a no-code approach - 99% of NiFi users will never see a line of code. It is a web based GUI with a drag and drop interface to build pipelines.
NiFi can perform ETL, and can be used in batch use cases, but it is geared towards data streams. It is not just about moving data from A to B, it can do complex (and performant) transformations, enrichments and normalisations. It comes out of the box with support for many specific sources and endpoints (e.g. Kafka, Elastic, HDFS, S3, Postgres, Mongo, etc.) as well as generic sources and endpoints (e.g. TCP, HTTP, IMAP, etc.).
NiFi is not just about messages - it can work natively with a wide array of different formats, but can also be used for binary data and large files (e.g. moving multi-GB video files).
NiFi is deployed as a standalone application - it's not a framework or api or library or something that you integrate in to something else. It is a fully self-contained, realised application that is fully featured out of the box with no additional development. Though it can be extended with custom development if required.
NiFi is natively clustered - it expects (but isn't required) to be deployed on multiple hosts that work together as a cluster for performance, availability and redundancy.
So, the two tools are used quite differently - hopefully that helps highlight some of the key differences
It's true that there is some functional overlap between NiFi and Camel, but they were designed very differently:
Apache NiFi is a data processing and integration platform that is mostly used centrally. It has a low-code approach and prefers configuration.
Apache Camel is an integration framework which is mostly used in distributed solutions. Solutions are coded in Java. Example solutions are adapters, flows, API's, connectors, cloud functions and so on.
They can be used very well together. Especially when using a message broker like Apache ActiveMQ or Apache Kafka.
An example: A java application is enhanced with Camel so that it can send messages to Kafka. In NiFi the first step is consuming those messages from Kafka. Then in the NiFi flow the message is changed in various steps. In the middle the message is put on another Kafka topic. A Camel function (CamelK) in the cloud does various operations on the message, when it's finished it put the message on a Kafka topic. The message goes through a NiFi flow which at the end calls an API created with Camel.
In a blog I wrote in detail on the various ways to combine Camel and Nifi:
https://raymondmeester.medium.com/using-camel-and-nifi-in-one-solution-c7668fafe451
What is mediation engine as referred to in camels documentation(below link)?
https://camel.apache.org/manual/latest/faq/what-is-camel.html
A use-case example too would be greatly appreciated.
The mediation engine reference in this context is originating from the topic of Enterprise Application Integration and is closely related to what GoF Mediator Pattern does - That is encapsulate communication between entities. In the case of EAI, a mediator/mediation engine sits between multiple disparate systems and acts as a broker between them, instead of letting the systems communicate directly.
Mediation approach in EAI offers capabilities like
Reduced coupling between systems. For instance, you do not have to learn and implement a legacy mainframe protocol in modern systems, just because you want to get some data from main frames. A mediation engine like Apache Camel could talk REST over HTTPS at one end and some archaic mainframe protocol on the other end.
Ease of migration: Once the mainframes area replaced with something else, you could just change the mediation layer to do something about it, instead of modifying multiple impacted systems that used to talk to mainframes.
Access to single resource/service via multiple channels: Let's say you have an old system that currently does SOAP over HTTP but you would like to offer REST with JSON payload to some of your new customers. Instead of building completely new systems up-front for this purpose, you could throw Apache Camel as a mediator and it would accept JSON payloads at one end and SOAP on the other end. Whoever wants to talk JSON can go through Camel and whoever wants to do SOAP may continue in a direct connection to the legacy system. Someday if some hypothetical FooBar protocol becomes popular, and if Apache Camel provides you a FooBar component, your users who demands FooBar support could be routed through Camel, to the system that still speaks SOAP.
All these stuff is discussed in detail in the Enterprise Integration Patterns site and book. Apache Camel implements truck loads of patterns as described in the EIP Book. I hope this answer helped you to understand the mediation role Apache Camel could play in Enterprise IT ecosystems.
From Camel in action
The core feature of Camel is its routing and mediation engine. A
routing engine selectively moves a message around, based on the
route’s configuration. In Camel’s case, routes are configured with a
combination of enterprise integration patterns and a domain-specific
language.
In this link, https://camel.apache.org/manual/latest/faq/what-is-camel.html indicated projects that can be components of the Camel route to and from where messages can be sent and consumed (https://camel.apache.org/components/latest/activemq-component.html, https://camel.apache.org/components/latest/cxf-component.html).
Apache camel is kind of a ESB middleware. Mediation with respect to Camel would mean the following
Data format transformation : If Application A speaks JSON and Application B understands CSV format. You can use Apache Camel to transfer JSON to CSV.
Protocol transformation : If Application A knows only to call webservices but Application B prefers reading data from a message queue. You can use Apache Camel to receive this data by exposing a webservice and then push it to a queue for application B to consume.
Content transformation - Filtering or Enriching data : During this transformation process, you can also transform the data by filtering or enriching data fields based on what application B needs. In this way no change is required in A as it sends what it has and no change is required in application B as it gets what it needs.
Connectors : Many ESB now have in-built connectors to connect directly with ERP or SAS based applications. For example, a kafka connector. https://camel.apache.org/blog/Camel-Kafka-connector-intro/
Any body is using Talend SOA Suite in their shop.
How does it compare with Oracle SOA Suite and Service Bus as both these products implement the EIP patterns if i am not wrong ?
Currently i am trying to study both but i am leaning towards Apache Camel and Talend Suite but is there anything that Oracle SOA suite and Oracle Service Bus provides that is over and above what Talend SOA suite does not.
Am i missing the bigger picture here ?
From my perspective, using the Oracle SOA/ESB Suite is a whole different ballgame in terms of feature scope, complexity, learning curve, price (obviously) and overall commitment to the Oracle stack...
Apache Camel is focused on a small subset of that...mainly EIPs to implement complex routing and components to ease integration with other technologies...that's it. Its not an ESB (see ServiceMix for that) or a container of any kind. Instead, its designed to be lightweight and easily embeddable in existing applications (Tomcat web apps, ActiveMQ, ServiceMix, standalone with Spring, etc). Overall, if you know Java, Maven & Spring, then you can get things up and running in literally minutes.
Camel also has a strong community following and has been embraced (along with ActiveMQ, Servicemix, CXF & Karaf) by a growing number of companies both large and small.
(full disclosure - I'm an Apache SOA Consultant, but I have some experience with the Weblogic stack as well)
Talend does offer a full ESB. That ESB use Apache CXF for SOAP and REST, Apache ActiveMQ for JMS, and Apache Camel for mediation and routing. Those three modules are provided in distribution that includes Apache Karaf. Karaf is an OSGI container. However, as boday pointed out you can run any or all of these components inside a Servlet, Java EE, or even a simple JVM instead if you prefer.
So that is the first difference, Talend is much more lighweight and modular. That modularity is much deeper than simple compliance to Java EE APIs. You can pick and choose whether you want CXF, ActiveMQ, or which Camel components in particular to include. If you opt for the Karaf / OSGI option you get to leverage the power of OSGI bundle management. This may not seem important at first glance, but it is quite important in an enterprise context.
OSGI manages dependencies at the package (rather than the jar) level. It is the same standard used by your Eclipse IDE. When you install a feature package from Eclipse Marketplace (e.g. subclipse) you are using OSGI feature management. That is why Talend include Apache Karaf in the distribution. So that you have the same pluggable, modular framework for both deployment and dependency management for your enterprise applications.
Why is that important? Because when you have different application teams deploying apps in the enterprise, you need a little bit of flex in your bus. You need to do a few transformations, a little bit of message correlation, use a few EIP's like idempotent consumer. Well all those transforms may need to act on the message payload or use jars provided by the project teams authoring the apps that are being integrated. And all those project teams are on different lifecycles. This is what Microsoft calls DLL hell, and that is why the invented .NET assemblies. Java does not have an equivalent feature that is part of the JCP. But there is a very mature standard that addresses this. That is OSGI. The open source engine that realizes OSGI is Equinox in Eclipse, or Apache Felix. Both are are pluggable in the Karaf container.
So much for the container, what about the heart of the ESB, Apache Camel. I'm sure most readers are aware, but Camel realizes the enterprise integration patterns (EIP) from the Gregor Hohpe and Bobby Woolf book of the same name. It is just a library, and it can run in any jvm, and in fact with or without DI container such as Spring. The important thing to take away is that Camel addresses the need for asynchronous, event driven integration which is necessary to address the fallacies of distributed computing. And it does it in a completely flexible way that does not require you to deploy Java EE servers everywhere. Want to have a dedicated integration server, great use Talend ESB out-of-the-box for an OSGI server. Want to service enable managed endpoints inside Tomcat, great, deploy Camel, CXF, and ActiveMQ inside Tomcat if you like. You get much greater, platform agnostic and architecture agility with a lightweight modular integration platform like Talend.
Another differentiator worth mentioning is that transformations are one of those mundane things that are still really important. You need to really think about how you plan to manage the schemas of the messages and events that are being placed on the bus. Talend ESB provides transformation tools OTB.
(full disclosure - I am Technical director at Talend, but I hope that I have emphasized the Apache and OSS aspects here in a very objective manner.)
Apache Camel is an integration framework. Talend and Oracle are ESB products respectively integration suites. Find a detailed description, comparison and recommendation when to use which one in my presentation slides: "Spoilt for Choice - How to choose the right Enterprise Service Bus"
(not just interesting for Talend and Oracle, but also for IBM, SAP, Spring Integration, Mule ESB, Fuse ESB, WSO2, and other integration frameworks / ESBs / integration suites)
[Like Ed, I also work for Talend, but this is a vendor-neutral presentation]
I'm looking for a high level technical gap analysis of the Apache ESB/SOA stack (Servicemix, Camel, ActiveMQ, CXF) vs. comparable Mule technologies.
As well, I'm trying to better understand how these frameworks are viewed amongst developers in terms of learning curve, stability, scalability and overall ability to meet client requirements...
It's not really an answer, but too long to be added as a comment.
Gartner does such comparisons (example), so does Forrester (example1; example2), but their papers are:
expensive to obtain
focusing more on the market share and the hype, less on the technical capability to deliver a solution
mainly about commercial products - maybe because market share for open source is difficult to measure (no licenses sold)
I personally have experience with Oracle Fusion (bad), Tibco (better) and Vitria (outdated), but I'm not up to the challenge to do a detailed comparison...
Camel uses a Java Domain Specific Language in addition to Spring XML for configuring the routing rules and providing Enterprise Integration Patterns
Camel's API is smaller & cleaner (IMHO) and is closely aligned with the APIs of JBI, CXF and JMS; based around message exchanges (with in and optional out messages) which more closely maps to REST, WS, WSDL & JBI than the UMO model Mule is based on
Camel allows the underlying transport details to be easily exposed (e.g. the JmsExchange, JbiExchange, HttpExchange objects expose all the underlying transport information & behaviour if its required). See How does the Camel API compare to
Camel supports an implicit Type Converter in the core API to make it simpler to connect components together requiring different types of payload & headers
Camel uses the Apache 2 License rather than Mule's more restrictive commercial license
Mulesoft Anypoint is a ready to use full-stack integration platform. Apache components functionally provide similar capabilities but generally take more time to implement and support. Both allow dropping down to Spring / Java level therefore no true technical gaps in either. The choice would depend on the business goals, available budget, and the scope and number of the integration projects. Mule offers better time to market and is easier to operate, but ain't particularly cheap. Apache stack is free but developers' time (generally) is not.
Camel is a EAI Framework and It doesn't have it's own runtime but other side Mule is full ESB product having it's own run-time. Mule has lot of connector to integrate with other system and stand itself as light weight ESB. Developers have full liberty to write own connector or invoke existing Java library to avoid rework.
I'm building a video streaming web site. I should receive video streams from several clients and I should broadcast that streams to subscribed(to a particular video) clients. Never did streaming before. I have no idea what protocol to use, what kind of software is required on server side and so on.
any suggestions would be appreciated.
This doesn't cover your requirements for receiving streams from clients, you'd have to do some further research on that.
However for the broadcasting from you to your subscribers, especially seeing as your using Silverlight, you definitly want to have a look at IIS Smooth Streaming to provide the streaming. It is a free addon to the IIS webserver and provides a stream to the subscriber based on what they can consume (based on bandwidth and computer capabilities).
When it comes to the player for the subscriber to view the stream, you might want to check out the Microsoft Media Platform (formerly known as the Silverlight Media Framework). Which is a prebuilt but fully customizable Silverlight player, which works well with Smooth Streaming.