I plan to add anti-virus protection to our web application that is being built. I have a concern that even the limited amount of files (PDF files, images, or even unknown binaries) that the user uploads may contain viruses.
Concerns:
The images are shared with other users (exposed to web pages) may contain viruses.
The PDF files that users share with each other may contain viruses.
The API that I build for this web application handles the file upload and this API is the file server as well.
Are there any state-of-the-art approaches to minimize the exposure of users to malware, including techniques in the API or techniques on the client-side (browser)? More specifically, I'm interested in solutions that would scan files in the API itself (backend). The files may be stored in a database or on the file-system.
I definitely searched Github for open-source tools and packages, moreover, ran several searches on Google against terms like "open source anti-virus API", "open-source malware HTTP API", but could not find any. Broader search terms resulted in a huge amount of unrelated results.
A related and outdated question investigates a similar problem, but I'm looking for a solution that would integrate well into a micro-service architecture, like Kubernetes, moreover, I think a canonical answer would be useful from an expert.
There are definitely solutions that can help you and integrate into your web application via an API. Here are a few that I am aware of:
SophosLabs Intelix
Intelix is a threat intelligence platform that provides access via APIs through AWS Marketplace. There are three parts to the service lookups, static analysis and dynamic analysis. Each one will give a more detailed analysis of the file. Combining the three will give you a good protection for your web application.
VirusTotal
VirusTotal is a community that will provide you with aggregated information showing you what various anti-malware vendors will say about your file. While VT is a great service, one thing to watch here is that VT is focused on being a community and therefore files uploaded are shared with others.
Clam AV
Not one that I have personal experience of but Clam AV allow you to spin up a server and then query it using API. There is a tutorial / documentation here.
Others
If you tweak your google search and look for Sandboxes most offer an API for a fee. A couple that come to mind Joe Sandbox, Falcon Sandbox which powers Hybrid Analysis.
As always, be careful of any cloud service that offers you scanning for free. Most of the free tools will share the reports and/or files within their community.
I already working on AppEngine which is my android backend but I have to create chat system for my app so I cannot figure out how to do that.
I'm using spring boot
please help. sorry for any kind of mistake.
You can use google compute engine on google cloud to write down your WebSocket server.
Also, you can use apache thrift for a seamless design of communication protocol between different language. It saves lots of repeated effort while designing communication protocol.
From Quora
There's a lot of repeated work you have to do when you're writing a server - primarily designing a protocol and writing code to serialize and deserialize messages on the protocol, but also dealing with sockets and managing concurrency, and writing clients in many languages. Thrift automatically does all of this, given a description of the functions you want to expose from your server to clients. It's also useful for serializing data on disk or into shared memory (where many of the same problems come up).
As a personal hobby, I would like to program a web-based card game with a few tokens and write an AI for it. I do not want to spend time and effort on standard elements such as maintaining a list of games and coordinating who's playing who, or even writing a login system (ideally I'd like to use Google accounts).
My choice of programming language is flexible, but would prefer something I could run on Google app engine.
I know Google Play Games provides some of the APIs but I was hoping for something more comprehensive. Even better if it works with Google Play Games.
Can you recommend toolkits that provide all or most of this functionality?
Board Game Arena supplies the community and lobby for your online board game, and also provides hosting and the community of players, and helps deal with licensing. The big downside is that you must comply with their system, and must write in PHP, and they don't work with Google accounts.
That said, it is a solution for the problem presented in the question, at least in some cases.
Though I am sure this is a little less fully formed than you were hoping, I would propose WT Toolkit, which allows for javascript-less C++ web applications.
Does it support a login system? Yes it does! supporting both google and facebook, with an easy path to integrationg other OAuth methods (hotmail for example)
List of games? std::vector
List of current people playing others? std::unordered_map
Games are closer to native apps than they are to web pages; a framework that allows you to leverage typical game design methods and exposed WebGL through a unified interface, like WT does, might make it easier for you by allowing you to focus on the GAME, not the WEB.
Maybe not complete answer but at least it didn't belong to comments. (Doesn't have to be correct)
On Google App engine things that will help:
Users service Will help you with authentication
Oauth on Appengine
The Channel API can help you with realtime sync between players and server
The endpoint's will help you with endpoints for devices if needed
XMPP service offered for sending and receiving chat messages
The above will get you started with a simple game. Recommend to look at the channel api for the tic tak toe game.
I hope it helps
I'm trying to build a car which can drive itself using video frames from an Android camera on-board using Neural networks. I haven't bought the hardware (car + android).
Meanwhile, I want to simulate the driver application (on my laptop) by streaming video frames from a 3D simulator and see if works properly.
What's an easy-to-learn and mostly pre-configured engine for this?
I think there are no easy-way to do that. In our robotic laboratory we use ROS to do this type of simulations.
It includes a 3D simulator (Gazebo), camera and sensor simulation and a lot of other stuff. You can see an example of 3D robot/camera simulation in this video.
Unfortunately ROS is not easy-to-learn and there is no complete documentation.
Another possibility is to use Blender (the open source, cross platform suite of tools for 3D). There are some robotic project that use Blender (you can see this page for more info) but I have no personal experience with it.
You can try out V-Rep, even though its a commercial product its much superior in performance. It has built in models for autonomous cars. If you are from India, contact Asimov Robotics, they are the resellers
To update this answer for 2017, Gazebo appears to be one of the leading 3D Simulation engines available today. Definitely worth looking into for this kind of application.
An excerpt from their home page:
Robot simulation is an essential tool in every roboticist's toolbox. A
well-designed simulator makes it possible to rapidly test algorithms,
design robots, perform regression testing, and train AI system using
realistic scenarios. Gazebo offers the ability to accurately and
efficiently simulate populations of robots in complex indoor and
outdoor environments. At your fingertips is a robust physics engine,
high-quality graphics, and convenient programmatic and graphical
interfaces. Best of all, Gazebo is free with a vibrant community.
http://gazebosim.org/
Also, the game Grand Theft Auto has been used to test self driving car algorithms. There are APIs to do everything from change the pedestrian traffic, to lights, to weather and more.
I have to prepare a comparison between the following technologies to present it to my Project Manager, but I fell that I'm lost, so if any one can help I will be thankful
I want to compare between them in the following areas:
the support of online video streaming
the budget of using each one
Learning Time will be needed to learn the technology
Which one is the standard and will target a lot of users
The support if I found any problem
Bugs and security issues
connection to DB, SOA and web services
supporting of multi player
The support of online video streaming
Some of the X3D viewers support video streaming (and some even 3D streaming, for things such as augmented reality).
Which one is the standard and will target a lot of users
X3D is a standardized format, such as JPEG with multiple companies being able to manipulate such data and is even officially recommended by HTML5 specs whereas Unity ties you to a single company. Even if most X3D viewers are plugin-based like Flash, there exists also native implementations such as X3DOM to display/interact with X3D files for any browser that supports WebGL.
Connection to DB, SOA and web services
I would usually recommend using a webservice for interfacing with a DB, and yes, X3D can interact with webservices (XML, JSON). There is even a standard binary format that is fast to transfer and parse large contents faster.
Supporting of multi player
Some X3D-supporting providers offer a multiusers service, such as Bitmanagement's BS Collaborate server, but I've seen people using Darkstar/RedDwarf to make multiusers 3D environments as well.
the support of online video streaming
Unity 3D does not support video streaming, unless done through textures, which will give you a really slow frame rate.
I don't know for sure about X3D, but I would doubt it was really made for such tasks.
Silver light has good video support, it should be easy to stream with.
HTML only supports streaming video if using HTML 5, for which it gives the best user experience when user's browser supports it.
Flash is the de-facto for video streaming. It is extensively widespread. They use it for YouTube for example.
the budget of using each one
The cheapest of them all is HTML, it is free. Then you can theoretically set up something for free in flash using Flex SDK and server streaming technology such as Red5 (both open source and free). After that, I believe that all others would probably be on par cost wise, Unity3D coming in as the cheapest of the paid alternatives.
Learning Time will be needed to learn the technology
Listed in order of fastest one to learn to slowest (assuming no prior experience in any):
HTML
Flash/Silverlight
Unity3D
X3D
Which one is the standard and will target a lot of users
Flash is the most widespread. Its only competitor would be HTML 5, as new browsers tend to support it and its the only possible option on iOS. On the other hand, if 3D is what you want, then Unity3D is the standard for now, might be followed by HTML 5 in the future.
The support if I found any problem
Well, Unity3D would offer you good paid support, flash and silver light also (but only when you pay for streaming server licenses). HTML, X3D will not give you any support, but you can find a lot of information on the internet. There is also extensive information about Flash and Silverlight on the internet, but mostly Flash.
Bugs and security issues
All are pretty secure, I'm just not sure about X3D, but all others are comparable in term of security or bug issues.
connection to DB, SOA and web services
Easy to do with HTML, Flash and Silverlight. Harder with Unity3D, and hardest with X3D.
supporting of multi player
Multi-player what? If you are making a game, then clearly I would say your real options are Unity3D if the game is to be in 3D, Flash if it is to be done in 2D. Check out SmartFoxServer for easy multiplayer server.
My 2 cents:
the support of online video streaming:
Some X3D players do support it. Unity does in some ways : http://unity3d.com/unity/features/audio-and-video
the budget of using each one:
X3D and Unity3d are free. You can pay for Unity licenses for extra features and platforms like iOS and Android. If you need to write plugins for Unity, you'll need the $1500 license. There are no costs for distribution of Unity products.
Learning Time will be needed to learn the technology:
Both X3D and Unity3d have active communities and many online resources and offline books. Unfortunately for X3D, the best content creation tool (Vivaty Studio) is no longer supported officially, but X3D is supported in Maya, Max, Blender, and many other 3D programs. Unity's online docs are excellent and the answers.unity3d.com forum (and other forums) are free and fast.
Which one is the standard and will target a lot of users:
'Standard' Well, HTML is the broadest standard. X3D (if including VRML) is the oldest most widely used 3D standard. HTML you have. HTML5 is coming, 'real soon now' (I'm already turning blue). If you mean 'most readily available' the HTML is #1, Flash is #2 (as everyone has a browser, and most computers come with Flash installed already). Flash needs to be installed. Unity needs to be installed too, but it's at least as fast and easy to install as Flash, and it's gotten millions of downloads, so it's getting pretty pervasive. X3D requires a plugin (this should change sometime 'real soon now' with x3dom on HTML5), but the many X3D players are all a little different from each other.
The support if I found any problem:
All have much online community support. X3D has a spec committee but that's not really support per se, you'd have to contact the X3D plugin provider (Bitmanagement, Cortona, Octaga, Exit Reality, Fraunhoffer, etc.) Unity has great online community forums, you can pay for premium support, but I'd only do that if I needed a serious bug or feature that has no work-around.
Bugs and security issues:
X3D's bugs depend on which player you use. Unity has bugs, but the product is pretty solid (I've only crashed it once, and I use is all day, every day, for over a year). Both have a mind toward security, but neither of these are totally secure, especially since you can write scripts that are inherently not secure. So you have a hand in how secure YOUR content will be. Some X3D players support encryption. Unity products are compiled.
connection to DB, SOA and web services:
You can use something like AJAX or JSON or whatever in all these platforms, no? So if it's by web service, sure. If by direct local access, I know Unity can do that. Both Unity and Flash require cross-server xml files on the server to allow access cross-domain (in the web player for Unity anyway).
supporting of multi player:
Unity has excellent multi player networking components. X3D (spec) supports it too, but it really depends on which X3D player you go with as to how well it actually works. Worst case, you can use AJAX or JSON or whatever to roll your own.
Which you choose depends mostly on what you want to do with it. Flash is generally the best route right now, unless it's all about 3D, then I'd try Unity. But a year from now, HTML5 alternatives will begin to take over. Flash DOES support 3D, there are different ways it can be done. Vivaty had a full-featured X3D player written in Flash, so it can be done. There are several good 3rd party 3d plugins for Flash.
I totally agree with wildpeaks : )
Connection to DB, SOA and web services: easy to do with HTML, Flash and Silverlight. Harder with Unity3D, and hardest with X3D.
Reply: I think X3D is not hardest.
X3D(X3DOM) can interact with webservices (XML) as very easy in this example/tutorial
Flash supports hardware accelerated 3d, and comes out of the box with 3d support. In addition, there is the papervision library for more advanced 3d. Unity3d is also supported
as a flash library.
I would consider Flex as a real alternative to Flash. It has the same actionscript language, but uses a tag based syntax called MXML, similar to silverlight. Database remoting is extremely simple. You can access your .Net/Java/Php objects directly on the front end without having to deal with serialisation issues. All of the Flash libraries are accessible.
There is also the X3D player from instantreality.org, supporting video streaming & decoding, XMLHttp request via scripting and its free for non commercial usage.
Flash 3D isn't good 3D for any application of real-time 3D. It is 2.5D with some tricks.
X3D is easy to learn for simple things and harder as complexity goes up. It does have the advantage of being VRML with pointy brackets so the free content, examples and toolkits are easily found. I did comparison tests of the various players. BS Contact is the best for the ability to handle the most complex content with the fastest frame rate and rich color palette. Network support is still non-standard although XMLHTTP and database connections are easy to bolt on. As others have said, Instant Reality is coming on fast and supported by people with a deep understanding of the past implementations and future requirements.
The decision comes down to the project type. A simple comparison rating such as you are is misleading at best but thanks for giving it a shot. I've used VRML through all of its incarnations and now X3D for world building and now as a source for 3D models in video work in combination with Sony Vegas. For cost-benefit without the need to use very expensive modeling toolkits, it is the best of all the choices.