Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Welcome!
I very enjoyed programming artificial intelligence in my studies - neural networks, expert machines and other. But in work I develop mainly web applications.
And now I think about returning to such programming, maybe in hobby, or maybe in work. Are there areas where AI is commonly used in applications development and programmer with such skills can search work?
Or maybe I can sold some ideas to my boss and use AI to extend some of our applications.
What are you experience and ideas with using AI in applications?
I recently started reading the book Programming Collective Intelligence. It's an excellent book which discusses exactly what you are looking for - using AI techniques in web applications.
The book is written clearly, is easy to understand, explains everything in terms of real applications (it covers how some commonly used technology works: Google Pagerank, Amazons recommendation system, matchmaking websites, link recommendation systems, bayesian spam filters and more) and it uses actually useful examples using real data (ebay API, facebook API etc are used to collect data). In one chapter, it even explains how you can draw graphs (I mean the data structure, not bar/line/etc graphs) optimally (so that no nodes are too close together, minimum overlapping lines etc), which could be useful for, for example, mapping social networks.
I would recommend having a look at it and see the different ways AI can be applied to web applications.
As a counter-example, parsing data acquired from water testing equipment would probably be a bad place to use artificial intelligence:
The Daily WTF: No, We Need a Neural Network
Just a reminder for all of us to choose the right tool for the right job.
Neural networks are great for working on images, so one area of web applications you could use AI for would be identifying and/or manipulating patterns in images over large sets of data. For example, a site like Flickr or Facebook might have some interesting training material to identify people based on face or associating groupings of pixels (those being the features you work with) with certain items mentioned in captions or tags.
In terms of text manipulation, there's a lot of stuff, but it's usually icing on the cake for other web apps. I'm talking mostly in the areas of automatic completion in search bars and back-end things the user doesn't usually see, like automatic machine translation or improved search capability.
The problem with putting AI at the front of an application's offering is that usually, artificial intelligence is not a feature in and of itself, but rather a way of negotiating large data sets effectively without regular prompts from the designer. In general, a user will associate with an application on a one-to-one basis, and therefore judges it only on the quality of a relatively low number of responses.
Email spam filtering systems - definitely.
Any other security applications which need to spot patterns for malicious stuff.
You probably could analyze the behavior of the visitors of your web applications ; how do they navigate inside the website to provide a better, optimized interface. Now it depends on what kind of web applications you're working on. For on line shopping you can come with suggestions extrapolated from customers habits.
You can also detect "abnormal" behavior and fraud. Fraud and bot detection can take advantage of AI.
Forecasting, of course.
It has immense value for businesses (i.e.: inventory optimization) and is especially valuable in the time of global crisis.
Games do need AI.
Expert systems too.
Outside of games, I've seen very few commercial uses of AI.
It could, in theory, be very useful in industrial robotics and imaging, but those fields also tend to be very conservative, and uncomfortable with non-deterministic algorithms.
You might want to research what iRobot does, but even them use rather simple algorithms in their commercial robots.
In the area of cognitive architectures (e.g. Soar, ACT-R, etc), rather than concentrating on algorithms like A* and games, researchers investigate models of human behavior including decision-making, cultural interchange and learning. They often focus on cognitive plausibility, i.e. how close does a model track what a human would do, including timing, etc.
These systems tend to be strictly research-based with limited commercial applications. So far anyway. Military applications, especially for training, are fairly common though.
Image Processing for detecting cancer! (We actually code IEEE papers about it, creating the algoritms is way harder than coding them so we write papers about the performance of other papers)
Risk assessment is a pretty good case for neural networks, mostly because they're pretty good at pattern matching. Insurance and credit companies use them to some degree for determining the risk of a customer.
I have done some extensive research on using Artificial Neural Networks for classification of underwater sound sources. The algorithm seemed to work quite well, especially that I devoted a big portion of the work on figuring out what combination of fourier transform coefficient composed the best set for the classification (with Principal Component Analysis).
Anything (seriously):
http://highlevellogic.blogspot.com/2010/09/high-level-logic-rethinking-software.html
The High Level Logic (HLL) Open Source project is about finding and coding high level logic under which all the other AI (and in fact, all programming) fits. There are serious concrete ideas and code. HLL is already an application framework.
Related
I need help to chose a project to work on for my master graduation, The project must involve Ai / Machine learning or Business intelegence.. but if there is any other suggestion out of these topics it is Ok, please help me.
One of the most rapid growing areas in AI today is Computer Vision. There are many practical needs where the results of your Master Thesis can be helpful. You can try research something like Emotion Detection, Eye-Tracking, etc.
An appropriate work for a MS in CS in any good University can highlight the current status of research on this field, compare different approaches and algorithms. As a practical part, it makes also a lot of fun when your program recognizes your mood properly :)
Netflix
If you want to work more on non trivial datasets (not google size, but not trivial either and with real application), with an objective measure of success, why not working on the netflix challenge (the first one) ? You can get all the data for free, you have many papers on it, as well as pretty good way to compare your results vs other peoples (since everyone used exactly the same dataset, and it was not so easy to "cheat", contrary to what happens quite often in the academic literature). While not trivial in size, you can work on it with only one computer (assuming it is recent enough), and depending on the type of algorithms you are using, you can implement them in a language which is not C/C++, at least for prototyping (for example, I could get decent results doing things entirely in python).
Bonus point, it passes the "family" test: easy to tell your parents what you are working on, which is always a pain in my experience :)
Music-related tasks
A bit more original: something that is both cool, not trivial but not too complicated in data handling is anything around music, like music genre recognition (classical / electronic / jazz / etc...). You would need to know about signal processing as well, though - I would not advise it if you cannot get easy access to professors who know about the topic.
I can use the same answer I used on a previous, similar question:
Russ Greiner has a great list of project topics for his machine learning course, so that's a great place to start.
Both GAs and ANNs are learners/classifiers. So I ask you the question, what is an interesting "thing" to learn? Maybe it's:
Detecting cancer
Predicting the outcome between two sports teams
Filtering spam
Detecting faces
Reading text (OCR)
Playing a game
The sky is the limit, really!
Since it has a business tie in - given some input set determine probable business fraud from the input (something the SEC seems challenged in doing). We now have several examples (Madoff and others). Or a system to estimate investment risk (there are lots of such systems apparently but were any accurate in the case of Lehman for example).
A starting point might be the Chen book Genetic Algorithms and Genetic Programming in Computational Finance.
Here's an AAAI writeup of an award to the National Association of Securities Dealers for a system thatmonitors NASDAQ insider trading.
Many great answers posted already, but I wanted to add my 2 cents.There is one hot topic in which big companies all around are investing lots of resources into, and is still a very challenging topic with lots of potential: Automated detection of fake news.
This is even more relevant nowadays where most of us are connecting though social media and there's a huge crisis looming over.
Fake news, content removal, source reliability... The problem is huge and very exciting. It is as I said challenging as it can be seen from many perspectives (from analising images to detect fakes using adversarial netwotks to detecting fake written news based on text content (NLP) or using graph theory to find sources) and the possbilities for a research proyect are endless.
I suggest you read some general articles (e.g this or this) or have a look at research articles from the last couple of years (a quick google seach will throw you a lot of related stuff).
I wish I had the opportunity of starting over a project based on this topic. I think it's going to be of the upmost relevance in the next few years.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
I am a Computer Science student. I want to do an AI project for my 4th year with two other students. (It's a 5-year degree in my university so I can pursue the same project for two consecutive years if I want to). Our knowledge in AI is very basic at this moment since we'll be specializing in it these coming two years, so a very advanced idea will probably be hard to accomplish. We're not expected to research new untouched soils either, so the more resources the better.
I'm interested in ideas that can benefit people and not just applying algorithms and techniques. I want to do a masters after graduation, but I'm not sure in what field yet.
I'd love to do a medical application or a project that of some use to the handicapped.
Some projects that were already pursued at the university included a project to recognize breast cancer, and to teach sign language to the deaf.
I'm wondering:
1) what other ideas we can work on in those fields?
2) how much will my choice of graduation project affect my application for a masters degree?
3) Is a stocks price prediction expert system too advanced for us?
Thanks a lot.
1) what other ideas we can work on in those fields?
It's amazing to me how little imagination computer science students seem to have. Stackoverflow.com is rife with questions about first projects from beginners and students.
I think that using statistics and data in novel ways, like Peter Norvig's spell checker, would be most interesting and fruitful.
Dr. Peter Norvig is a well-known computer science professor and AI guru. He's the CTO of Google now. Perhaps you can mine a choice out of his writings.
2) how much will my choice of
graduation project affect my
application for a masters degree?
Depends on too many other factors that you don't mention, like your past record as a student, etc. Probably a minor factor, in my opinion. Nobody is admitted to a masters program on the basis of a graduation project. Neither your undergrad project nor a masters thesis is a doctoral dissertation. Don't get them confused.
3) Is a stocks price prediction expert system too advanced for us?
I think stock price prediction is too advanced for anybody. After years of applying Fourier analysis, statistical models, Monte Carlo simulations, etc. if it were possible to do it would have been done.
2) how much will my choice of graduation project affect my application for a masters degree?
If you are applying for a PhD, the faculty in the prospective department tend to favor students who are interested in the research they are doing, or who have demonstrated the ability to do their own research. For a Masters these are not much of an issue, but they can make a little difference.
3) Is a stocks price prediction expert system too advanced for us?
Well, if you did then you would start using it to make money, others would see what you are doing an imitate you so that pretty soon your arbitrage opportunity would be gone.
Still, these type of systems are often built by students in machine learning classes, mostly due to the fact that there is a lot of data freely available and well formatted data on stock prices, so its easy to get starting writing the program. It is a good way to get insight into machine learning algorithms.
1) What other ideas we can work on in those fields?
Find some problem that you are passionate about, will learn something from by tackling it, and is within the scope of your time, effort, and ability. Projects like this are relevant not only for grad school but also when applying for entry-level jobs (even if a few years off still after doing a masters degree)l. It helps to pick something you can put on a resume that shows your level of accomplishment and ability to complete a task.
2) How much will my choice of graduation project affect my application for a masters degree?
The topic choice probably won't matter significantly except perhaps for top-tier programs or if you have notable weaknesses in other admissions criteria. If the latter is true, then a good project may help, but even the latter is uncertain. Masters program admissions I think is generally handled by administrative staff, so they are probably more interested in whether or not you did a project than what the topic is.
3) Is a stocks price prediction expert system too advanced for us?
Yes, a stock price prediction system is far too difficult if you want a system that actually can work reasonably well over anything other than a small training data set.
The market is neither a natural system, a machine, nor even a system of rational collective behavior. Its pricing mechanism is in general irrational: investors/traders may make transactions at prices that are reasonable for them relative to their own decision criteria, but the market as a whole is generally not rational. The market is more an aggregation of behavior rather than collective behavior.
The above alone would make for an intensively difficult problem to solve with AI methods, but beyond that there are issues of problem scale, the amount of training data which is needed, etc.
There are of course a large number of Wall Street trading firms using quantitative methods for high-frequency trading, etc. They are effective, however, because they are focused on narrow problems (price trends over the next few seconds-to-minutes in highly-liquid stocks, S&P index futures, etc.), they put a lot of work into their models and generally are constantly rebuilding the latter on a daily/weekly basis, and they understand the market's nature, i.e., it's largely irrational as a whole and is a competitive, shifting landscape of exploiting the pricing inefficiencies inherent to large money flows.
I would only recommend this problem domain if you have an intense personal interest in financial markets and have already spent a lot of time studying them, are prepared to fail, and are interested in learning a lot. Trying to work on this problem is certainly a good learning opportunity, but it will be hard to achieve any real success except for small problems unless you have many years to devote.
1) what other ideas we can work on in
those fields?
Dr. Russel Greiner has a nice list of possible student projects in machine learning, several of which are related to medicine.
2) how much will my choice of
graduation project affect my
application for a masters degree?
It probably won't matter very much. However, choosing a ridiculously easy project probably won't help. I'm sure that you'll be vetting whatever you choose with your prof, so don't worry about that so much. Find a topic you're passionate about first and foremost.
3) Is a stocks price prediction expert
system too advanced for us?
Yes. Don't bother with that nonsense. The game of Go will be solved before anyone figures out the stock market.
1) what other ideas we can work on in
those fields?
Are there any faculty members at your university that work in the field of bioinformatics? If so, talk to them and see if they give you a suitable project idea that gets you excited. If you decide to take this path, try to enroll in an Intro to Bioinformatics course as it will help you get familiar with the field and generally make things easier.
There are many papers about ranged combat artificial intelligences, like Killzones's (see this paper), or Halo. But I've not been able to find much about a fighting IA except for this work, which uses neural networs to learn how to fight, which is not exactly what I'm looking for.
Occidental AI in games is heavily focused on FPS, it seems! Does anyone know which techniques are used to implement a decent fighting AI? Hierarchical Finite State Machines? Decision Trees? They could end up being pretty predictable.
In our research labs, we are using AI planning technology for games. AI Planning is used by NASA to build semi-autonomous robots. Planning can produce less predictable behavior than state machines, but planning is a highly complex problem, that is, solving planning problems has a huge computational complexity.
AI Planning is an old but interesting field. Particularly for gaming only recently people have started using planning to run their engines. The expressiveness is still limited in the current implementations, but in theory the expressiveness is limited "only by our imagination".
Russel and Norvig have devoted 4 chapters on AI Planning in their book on Artificial Intelligence. Other related terms you might be interested in are: Markov Decision Processes, Bayesian Networks. These topics are also provided sufficient exposure in this book.
If you are looking for some ready-made engine to easily start using, I guess using AI Planning would be a gross overkill. I don't know of any AI Planning engine for games but we are developing one. If you are interested in the long term, we can talk separately about it.
You seem to know already the techniques for planning and executing. Another thing that you need to do is predict the opponent's next move and maximize the expected reward of your response. I wrote a blog article about this: http://www.masterbaboon.com/2009/05/my-ai-reads-your-mind-and-kicks-your-ass-part-2/ and http://www.masterbaboon.com/2009/09/my-ai-reads-your-mind-extensions-part-3/ . The game I consider is very simple, but I think the main ideas from Bayesian decision theory might be useful for your project.
I have reverse engineered the routines related to the AI subsystem within the Street Figher II series of games. It does not incorporate any of the techniques mentioned above. It is entirely reactive and involves no planning, learning or goals. Interestingly, there is no "technique weight" system that you mention, either. They don't use global weights for decisions to decide the frequency of attack versus block, for example. When taking apart the routines related to how "difficulty" is made to seem to increase, I did expect to find something like that. Alas, it relates to a number of smaller decisions that could potentially affect those ratios in an emergent way.
Another route to consider is the so called Ghost AI as described here & here. As the name suggests you basically extract rules from actual game play, first paper does it offline and the second extends the methodology for online real time learning.
Check out also the guy's webpage, there are a number of other papers on fighting games that are interesting.
http://www.ice.ci.ritsumei.ac.jp/~ftgaic/index-R.html
its old but here are some examples
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 years ago.
Improve this question
In my career I've come across two broad types of theory: physical theories and educational/management theories:
Physical theories are either correct (under appropriate conditions) or incorrect, as judged by the physical world.
Educational/management theories have the appearance of being like physical theories, but they lack rigorous testing. At best they give new ways of thinking about problems. Multiple theories are useful because one of them may speak to you in the right way.
As an hobbyist student of software engineering there appear to be a lot of theories of software engineering (such agile programming, test driven design, patterns, extreme programming). Should I consider these theories to be physical-like or educational/management-like?
Or have I mis-understood software engineering and find myself being in the position of "not even wrong"?
Software engineering is ultimately about psychology, how humans manage complexity. So software engineering principles are far more like education and management theories than physical principles.
Some software engineering has solid math behind it: O(n log n) sorts are faster than O(n^2) sorts, etc. But mostly software engineering is about how humans think about software. How to organize things so that maintainers don't go crazy, anticipating what is likely to change and what is not, preventing and detecting human errors, etc. It's a branch of psychology or sociology.
I think the appropriate theoretical split is those "harder" sciences (where there can be proofs) and the softer topics with qualitative answers and few proofs if any.
Software to me is mostly about language and communication, a topic that is qualitative and subjective mostly. Every now and then we touch into algorithms and other "hard" areas, where proofs and rigorous formalisms exist. So, yes, both please.
Not even wrong.
All the software engineering "theories" seem to be nothing but advice on particular things to try to see if they make you and your team more productive. Even if one could set them up to be falsifiable like scientific theories, there would be very little point to it. That is not to say that it is not worthwhile to learn about them -- au contraire, you should become familiar with as many of them as possible and try to figure out in what kinds of teams and environment they may work better. But be careful: avoid dogma and thinking there are silver bullets.
i wouldn't call agile programming, test driven design, patterns, extreme programming, etc 'theories', they're methodologies, or work styles. they don't make any assertions.
Generally the field of Informatics is divided into 4 areas (need to find a link to the source, SWEBOK?), which are distinct although related and interconnected:
Computer Science
Software Engineering
Computer Engineering
Information Systems
There is a good analysis of engineering vs. science in Steve McConnel's "Professional Software Development". Check out his Software Engineering, Not Computer Science.
Software development is more about engineering - finding practical solutions to practical problems - than anything else. That is correct that software engineering relies on computer science, mathematics, complexity theory, systematics, psychology, and other disciplines, but it can not be equated to any of them, nor is it a batch of any of them.
Besides theories, there are also frameworks, models and rules of thumb. Ideas, sure, but based on a less rigorous foundation, which loosely belong to your eduction/management category.
Computer Science has some strong foundational theories (physical ones by your definition), but these mostly consist of tying together the smaller elements.
Software Engineering on the other hand, is a relatively new discipline that involves utilizing computers and occasionally Computer Science to build software systems. Most of the practice in that arena is entirely based on non-rigorous experimental and anecdotal evidence. Since the jury is still out on even the simplest of issues, most of what passes for practices could be best described as pure guess-work and irrational preference. It's one of those disciplines where you really do have to know a lot to realize how much is built on a house of very unstable cards.
Paul.
Being intangible, programming is a very difficult activity to relate to another human being, even other programmers. Software engineering tries to add structure where there is none, but such structure is not rooted in the inevitability of reality. So all these approaches become like religions in how groups of people behave when trying to appease their technical gods (or demons).
All these theories and best practices still haven't brought us to the point where we can produce software systems reliably and predictably. The newest of these surveys is dated 2001; Jeff's column from 2006 still laments high failure rates.
It'd be interesting to see if anybody's working on an updated survey.
Avionics and the software running my car don't seem to fail at anything close to the rates quoted for enterprise software. Why don't enterprise developers follow their practices more closely? Maybe we should all be writing Ada....[just kidding]
They're like recipes: they're guidelines, whose success depends:
Partly, on the quality of the recipe
Partly, on the quality of the ingredients
Partly, on the skill of (and time available to) the practitioners
For me, it's my own theory with many of the others used as a base. I don't know any one that uses a single specific theory. And that's not a cop out answer.
Just as there are different languages, theories/practices/methodologies are to be used in distinct situations. The structure, rules, and definitions are all the ways in which people understand how things are to be accomplished, but what is to be accomplished is subjective.
Adapt, knowing the agile, extreme, or other methods at the discretion of the client, project, programmer, time, and especially what makes you successful/happy. Be a team and adjust/adapt to what your team is doing for the greater good; just keep in mind to have something that you have defined in your own mind, or it's not just chaos.
[SOAPBOX]
I started programming on the Atari 400 with a converted flat keyboard and 64K upgrade. When I started college, it was VB 1.0 which I saw my Economics Teacher use to build a teaching tool to help people learn more about economics using graphs and visual inputs. That was cool! And I knew I could do that.
This same Economics Teacher, who later become an IT teacher too (he was good), asked if I would teach a class on debugging. He said, "I haven't met someone that understands the concepts and has a natural ability to debug as fast as you do, would you teach us what you know and how you do it." This was a boost in my ego, of course, but to teach, mentor, and help others.
Every one of those instances has fuled my desire to help other people. For me, I want a computer to do exactly what I want, to help other people in the business and home life to increase their qualify of living, learn more, and get more done.
Someone said to me one time, "You're only as good as your tools". Learn, practice, and grow.
If you've defined something, it's working, has order, and it stretches you and the boundaries, you're not wrong.
Is there a think like "software engineering"?
Or Is software development is "engineering"?
Facts:
Our industry is very young realtive to many other "engineerings".
We still does not have "solid" practices and "theories".
So to be honest, if we look in the other mature engineering practices perpective, it is hard to call what we do as "engineering"
We have a very bad reputation at failing [ our failing rates can not be acceptable at many engineering branches]
Reality or Fantasy? Pick one :-)
Some guys say that we do not have "solid" paractices and "theories" since we are a young "engineering" branch, by time we will have.Those guys say that wee need to work more "theory" or foundations.
Some guys say that software develepment is "experimental social activity" because of the nature of our problem domain. We will have practices theories methodologies processes but they will always have second order effect. The unique people, their feelings qualities and and their interactions with the rest are more influential. Those guys see software development as Complex Adaptive System
And there is also another reality
%80 of software development activities really do not need very brilland mind. Any "average" person can do it.
But remaining %20 part is ver hard and multidiciplinary task.
Even there is an another new perspective My One:-)
This view say that software development is not a branch of "Engineering". It is brach of "Natural Sciences and Social Sciences". So we need Software Anthropology and Anthropologist.
Theory: I think a theory is anything that describes "how" a natural system works, and in order to prove it, has logical deductions based on previous knowledge, substantiated by logical inductions made using experiments.
You call the whole body of these theories and experiments as Science.
Software: Software is a man-made system aka. an engineered system. Engineering applies Science in order to create the new systems. In that regards, pure Software Engineering applies the science of discrete mathematical systems.
But Commercial Software Engineering has a different motivation called Economics.
In that regards, it has to take all the factors into account that affect Economics, the chief of them being People. So, Psychology plays a huge part .
But, since Psychology itself is just a theory of "how" human mind works based on just pattern recognition without any logical deductions based on human biology, it has many flaws like correlation implies causation.
So, yeah, I think from the above answer, you can better understand what Commercial Software Engineering in total is .
I've been working on a project, which is a combination of an application server and an object database, and is currently running on a single machine only. Some time ago I read a paper which describes a distributed relational database, and got some ideas on how to apply the ideas in that paper to my project, so that I could make a high-availability version of it running on a cluster using a shared-nothing architecture.
My problem is, that I don't have experience on designing distributed systems and their protocols - I did not take the advanced CS courses about distributed systems at university. So I'm worried about being able to design a protocol, which does not cause deadlock, starvation, split brain and other problems.
Question: Where can I find good material about designing distributed systems? What methods there are for verifying that a distributed protocol works right? Recommendations of books, academic articles and others are welcome.
I learned a lot by looking at what is published about really huge web-based plattforms, and especially how their systems evolved over time to meet their growth.
Here a some examples I found enlightening:
eBay Architecture: Nice history of their architecture and the issues they had. Obviously they can't use a lot of caching for the auctions and bids, so their story is different in that point from many others. As of 2006, they deployed 100,000 new lines of code every two weeks - and are able to roll back an ongoing deployment if issues arise.
Paper on Google File System: Nice analysis of what they needed, how they implemented it and how it performs in production use. After reading this, I found it less scary to build parts of the infrastructure myself to meet exactly my needs, if necessary, and that such a solution can and probably should be quite simple and straight-forward. There is also a lot of interesting stuff on the net (including YouTube videos) on BigTable and MapReduce, other important parts of Google's architecture.
Inside MySpace: One of the few really huge sites build on the Microsoft stack. You can learn a lot of what not to do with your data layer.
A great start for finding much more resources on this topic is the Real Life Architectures section on the "High Scalability" web site. For example they a good summary on Amazons architecture.
Learning distributed computing isn't easy. Its really a very vast field covering areas on communication, security, reliability, concurrency etc., each of which would take years to master. Understanding will eventually come through a lot of reading and practical experience. You seem to have a challenging project to start with, so heres your chance :)
The two most popular books on distributed computing are, I believe:
1) Distributed Systems: Concepts and Design - George Coulouris et al.
2) Distributed Systems: Principles and Paradigms - A. S. Tanenbaum and M. Van Steen
Both these books give a very good introduction to current approaches (including communication protocols) that are being used to build successful distributed systems. I've personally used the latter mostly and I've found it to be an excellent text. If you think the reviews on Amazon aren't very good, its because most readers compare this book to other books written by A.S. Tanenbaum (who IMO is one of the best authors in the field of Computer Science) which are quite frankly better written.
PS: I really question your need to design and verify a new protocol. If you are working with application servers and databases, what you need is probably already available.
I liked the book Distributed Systems: Principles and Paradigms by Andrew S. Tanenbaum and Maarten van Steen.
At a more abstract and formal level, Communicating and Mobile Systems: The Pi-Calculus by Robin Milner gives a calculus for verifying systems. There are variants of pi-calculus for verifying protocols, such as SPI-calculus (the wikipedia page for which has disappeared since I last looked), and implementations, some of which are also verification tools.
Where can I find good material about designing distributed systems?
I have never been able to finish the famous book from Nancy Lynch. However, I find that the book from Sukumar Ghosh Distributed Systems: An Algorithmic Approach is much easier to read, and it points to the original papers if needed.
It is nevertheless true that I didn't read the books from Gerard Tel and Nicola Santoro. Perhaps they are still easier to read...
What methods there are for verifying that a distributed protocol works right?
In order to survey the possibilities (and also in order to understand the question), I think that it is useful to get an overview of the possible tools from the book Software Specification Methods.
My final decision was to learn TLA+. Why? Even if the language and tools seem better, I really decided to try TLA+ because the guy behind it is Leslie Lamport. That is, not just a prominent figure on distributed systems, but also the author of Latex!
You can get the TLA+ book and several examples for free.
There are many classic papers written by Leslie Lamport :
(http://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html) and Edsger Dijkstra
(http://www.cs.utexas.edu/users/EWD/)
for the database side.
A main stream is NoSQL movement,many project are appearing in the market including CouchDb( couchdb.apache.org) , MongoDB ,Cassandra. These all have the promise of scalability and managability (replication, fault tolerance, high-availability).
One good book is Birman's Reliable Distributed Systems, although it has its detractors.
If you want to formally verify your protocol you could look at some of the techniques in Lynch's Distributed Algorithms.
It is likely that whatever protocol you are trying to implement has been designed and analysed before. I'll just plug my own blog, which covers e.g. consensus algorithms.