I have been fascinated by Minsky's "Society of the Mind" for now close to two decades. However, I just realized that I have not come across any general implementation of the model (and preferable an implementation that is accessible and in the open source).
I recently ran into this article by Push Sing (now tragically deceased, student of Minsky), http://web.media.mit.edu/~push/ExaminingSOM.html where he also notes that such an implementation does not exist.
I wonder if someone knows differently and if such a project or corpus of software does exist.
Note: I am aware of SOAR, ACT-R, Cyc, etc.
Thanks.
I think that Minsky's "Society of the Mind" model is more literature and aspirational than an actual model. It has lots of systems, information flows and controls, but there isn't anything that you could implement without doing a lot of invention. Once you put it all together, you'd have a big data flow network, but you wouldn't have any data, so you would then need to fill it with representations of facts, concepts, connections, and so on. Then you would need to figure some way of having inputs and outputs that are understandable.
Let's say you built it --- what then? There is no obvious way to test it. It's unlikely that you would suddenly get consciousness pouring out. The only way to drive new action would be to feed new input, but it's unclear what sort of output would result. Minsky would probably smile and say that the same is true of a human baby. But at least with children we have a good idea that they will being to exhibit some kind of intelligence beyond stimulus response.
Still, if you have the time, why not give it a try?
Related
I've been studying hierachial reinforcement learning problems, and while a lot of papers propose interesting ways for learning a policy, they all seem to assume they know in advance a graph structure describing the actions in the domain. For example, The MAXQ Method for Hierarchial Reinforcement Learning by Dietterich describes a complex graph of actions and sub-tasks for a simple Taxi domain, but not how this graph was discovered. How would you learn the hierarchy of this graph, and not just the policy?
In Dietterich's MAXQ, the graph is constructed manually. It's considered to be a task for the system designer, in the same way that coming up with a representation space and reward functions are.
Depending on what you're trying to achieve, you might want to automatically decompose the state space, learn relevant features, or transfer experience from simple tasks to more complex ones.
I'd suggest you just start reading papers that refer to the MAXQ one you linked to. Without knowing what exactly what you want to achieve, I can't be very prescriptive (and I'm not really on top of all the current RL research), but you might find relevant ideas in the work of Luo, Bell & McCollum or the papers by Madden & Howley.
This paper describes one approach that is a good starting point:
N. Mehta, S. Ray, P. Tadepalli, and T. Dietterich. Automatic Discovery and Transfer of MAXQ Hierarchies. In International Conference on Machine Learning, 2008.
http://web.engr.oregonstate.edu/~mehtane/papers/hi-mat.pdf
Say there is this agent out there moving about doing things. You don't know its internal goals (task graph). How do you infer its goals?
In way way, this is impossible. Just as it is impossible for me to know what goal you had mind when you put that box down: maybe you were tired, maybe you saw a killer bee, maybe you had to pee....
You are trying to model an agent's internal goal structure. In order to do that you need some sort of guidance as to what are the set of possible goals and how these are represented by actions. In the research literature this problem has been studied under the terms "plan recognition" and also with the use of POMDP (partially observable markov decision process), but both of these techniques assume you do know something about the other agent's goals.
If you don't know anything about its goals, all you can do is either infer one of the above models (This is what we humans do. I assume others have the same goals I do. I never think, "Oh, he dropped his laptop, he must be ready to lay an egg" cse, he's a human.) or model it as a black box: a simple state-to-actions function then add internal states as needed (hmmmm, someone must have written a paper on this, but I don't know who).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
[Objective-C]
Do you still use Styrofoam balls to model your systems, where each ball
represents a class?
Tom Love: We do, actually. We've also done a 3D animation version of
it, which we found to be nowhere near
as useful as the Styrofoam balls.
There's something about a physical,
conspicuous structure hanging from the
ceiling right in the middle of a
development project that's regularly
updated to provide not only the
structure of the system that you're
building, but also the current status
of each one of the classes.
We've done it on 19 projects the last time I've counted. One of them was 1,856 classes, which is big - actually, probably bigger than it should be. It was a big commercial project, so it needed to be somewhat big.
Masterminds of Programming
It is the first time I've read or heard about using styrofoam balls to model classes.
Is that a commonly used technique? And, how does that sort of modeling help us to design better the system?
If you have any photos to share which can show us how the classes are represented it'd be great!
Update: So, it seems that the material most people use is the paper. Styrofoam balls are actually oddballs, not a commonly used technique.
Noticeable techniques:
"paper plates and string" modeling, NealB
Post-it Notes on a whiteboard, Jason
Class-Responsibility-Collaboration cards, duffymo
Sheets of ruled paper taped to the wall, AMissico
Thank you all for the very good answers.
I found a couple of styrofoam models for:
Windows 95
and
Lotus Notes
(if that helps)
Actually, here's a Tom Love case study that shows a couple of his models.
This model may represent the least
expensive CASE tool on the market --
materials cost $20.35. It was more
useful than any CASE tools I have ever
used.
We used it in three important ways.
It fixed the number of classes that we would deliver in the finished
application and we did not allow new
ones to be added, unless existing ones
could be removed.
It was a very useful way to publicly document which classes had
been code reviewed (blue ribbons) and
tested (green ribbons).
It helped everyone understand what was being built and how much time and
effort it takes to do testing,
documentation and code reviews.
Edit: photo of object model
alt text http://img686.imageshack.us/img686/82/stryrofoamobjectmodel.jpg
The styrofoam ball model appears to date back to the mid 1990's - a time when CASE (Computer Aided Systems Analysis)
systems were all the rage.
At that time CASE systems promised significant benefits but were dismally slow,
buggy, unstable, overextended and downright awkward to use. Basically, long on potential but short on delivery.
I remember having a conversation with an analyst working on a different project from mine. Her team had
become so frustrated with their CASE system that they trashed it and resorted to "paper plates and string"
modeling. They reserved a meeting room, removed all the furniture, and laid out their process model using labeled
paper plates with strings (representing data flows) connecting them. She claimed it was much more
useful than the CASE system it replaced.
I suspect that the styrofoam ball model had similar roots.
Using styrofoam balls or paper plates fostered design "buy-in". If a team
finds something to rally around it naturally creates a common design focus. It is simple, concrete and
minimal - using it requires a lot
of face to face interaction and discussion. And that is where the value comes from. I suspect
if you brought a new person into the project and told them to bring themselves up-to-speed by
reviewing the "model" they would be "dead in the water". However, walk them through the
"model" and a real conversation would occur where all the required information need to
perform on the project would be imparted very quickly and efficiently.
Do I think styrofoam balls could become a sustainable modeling tool? No, I don't. They would be a real
pain to keep up to date in a changing environment. They convey little information. There are better tools available
today. And most importantly, if the team you are working with don't "buy" it, and they
probably won't, it will look really stupid - kind of like a sports team mascot, a rallying point
only if the team "buys it".
No, we don't do this. And in my 30-odd year history in the IT industry, I've never heard of anyone doing this.
The only way this could help you design better systems is by:
keeping the class count down since it's hard to build the styrofoam model; and
minimising changes, since updating it would be a serious pain in the rear end.
Other than those two dubious features, I can't see this as being very useful. I'd almost conclude it was some sort of prank. Far better to do some real work, I think.
Seriously, if we tried to model our application with styro coffee cups and straws, our bosses would be calling the men in white coats.
Post-it Notes on a whiteboard seem to be popular in the circles I travel in. Objects go on the Post-Its, and you rearrange them until you get your relationships the way you want em.
And then there are the Color Modeling people who use a 4-pack of colored Post-Its and assign an archetype to each color. It doesn't sound like this is much of an improvement, but standing across a room looking at it, you can tell where there are missing features or unidentified objects in the system.
There is one application to this that I think we tend to forget-- using tools to articulate an architecture comes naturally to us after years in the industry, but there are valuable, albeit less technically-minded, stakeholders who may not grasp vital concepts as readily. It would sometimes be a lifesaver to point to a cluster of balls and say, "This is the Language Processing Model, and if I implement the feature you want, it will have consequences here, here, and here. You can see that there are a lot of balls connected there".
Architects, be they designing buildings or systems, might rely on those tangible models to indoctrinate the check writers into the process.
And I thought that UML was useless. The styrofoam ball model makes UML look positively elegant by comparison.
Ward Cunningham's CRC card idea is more useful, even cheaper, and still retains that tactile quality that Dr. Love was after.
I had never heard of the idea until I read this question. It deserves an up vote for originality. And the "Windows" and "Lotus Notes" pictures are priceless.
Sheets of ruled paper taped to the wall, where each sheet is a component, class, entity, or whatever is needed. Everyone has a pencil.
Everyone can write on them "flushing" out the model during the design meetings. Such as, meeting notes, implemetation notes, new classes, removed classes, reasons why you do not have a particular class, and so on. After the design meeting, the principal designer takes them down and rewrite them, again "flushing" them out with pen in "rough-draft" versions. The designer can then make decisions based on the notes of each sheet, create new sheets for any additional components. Generate topics for next meeting, note any descrepancies, note any design / implementation details needed for coding, or whatever else they need to do.
Repeat the meetings until everyone is satisfied. Pencil is new stuff, pen is previous items. Once everyone is happy, the designer creates the working-draft, and posts where everyone can see and initial, in pen, their acceptance of the "working-draft".
Nothing is final. Pen versions are "latest" versions. Pencil versions are "work-in-progress" or "draft" versions.
Simple, fast, flexible, no wasting time on the computer, with high visiblity. Working man's Wiki.
No. My team does not do this.
And I am badly tempted to mock with image macros. But I'm contemplating that the idea is silly enough that it is self-mocking.
Say I come up with some super-duper way of representing some data that I think would be useful for other people to know about and use. Assume I have a 'spec' in some form, even if it might not be a completely formal one: ie, I know how this file format will work already.
How would I then go about releasing this spec to get comments and feedback based on it? How would I get it 'standardised' in some form?
Specifying file formats is difficult. If the data you want to store is trivial, it tends to be trivial. In general however, this is hardly the case. You can use the RFC structure and keywords, but I always found specifying a fileformat in prose a slow, difficult and boring task, also because reading it is likewise difficult.
My suggestion, if you want to follow this way, is to focus on blocks of information. Most of the difficuly is for entities that are optional, and present only if another condition happens, so try to exploit this when partitioning your data.
The best spec, IMHO, is real code with an uberperfect testsuite.
As for standardization, if enough people use it, it becomes a de-facto standard. you don't need an official stamp for it, although when the format is used enough, you could benefit from an official mime type.
To talk about it, well, it depends. I found useful to talk in terms of "object oriented" entities, and also in terms of relationships. Database-like diagrams are very useful on this respect.
Finally, try to find a decent already standard alternative first, or at least try not to deal with the raw bits. There are a lot of perfect container formats out there that free you of many annoying tasks. The choice of the container depends on the actual type of file format (e.g. if you need encryption, interleaving, transactions, etc).
There are a couple of ways I'd go about it, I think.
First, determine if there's a standards body (like W3C, or IEEE) that might be related to your file format. If there is, pitch it to them. I have no idea how receptive they'd be though.
Second, a standard is useless if nobody is using it. Get some momentum behind it. Write a blog post, twitter and make a website about it. Link on programming.reddit.com, and slashdot. Describe it to your friends and colleagues. Post it here on SO, and ask for feedback.
HTH.
A friend of mine is beginning to build a NetHack bot (a bot that plays the Roguelike game: NetHack). There is a very good working bot for the similar game Angband, but it works partially because of the ease in going back to the town and always being able to scum low levels to gain items.
In NetHack, the problem is much more difficult, because the game rewards ballsy experimentation and is built basically as 1,000 edge cases.
Recently I suggested using some kind of naive bayesian analysis, in very much the same way spam is created.
Basically the bot would at first build a corpus, by trying every possible action with every item or creature it finds and storing that information with, for instance, how close to a death, injury of negative effect it was. Over time it seems like you could generate a reasonably playable model.
Can anyone point us in the right direction of what a good start would be? Am I barking up the wrong tree or misunderstanding the idea of bayesian analysis?
Edit: My friend put up a github repo of his NetHack patch that allows python bindings. It's still in a pretty primitive state but if anyone's interested...
Although Bayesian analysis encompasses much more, the Naive Bayes algorithm well known from spam filters is based on one very fundamental assumption: all variables are essentially independent of each other. So for instance, in spam filtering each word is usually treated as a variable so this means assuming that if the email contains the word 'viagra', that knowledge does affect the probability that it will also contain the word 'medicine' (or 'foo' or 'spam' or anything else). The interesting thing is that this assumption is quite obviously false when it comes to natural language but still manages to produce reasonable results.
Now one way people sometimes get around the independence assumption is to define variables that are technically combinations of things (like searching for the token 'buy viagra'). That can work if you know specific cases to look for but in general, in a game environment, it means that you can't generally remember anything. So each time you have to move, perform an action, etc, its completely independent of anything else you've done so far. I would say for even the simplest games, this is a very inefficient way to go about learning the game.
I would suggest looking into using q-learning instead. Most of the examples you'll find are usually just simple games anyway (like learning to navigate a map while avoiding walls, traps, monsters, etc). Reinforcement learning is a type of online unsupervised learning that does really well in situations that can be modeled as an agent interacting with an environment, like a game (or robots). It does this trying to figure out what the optimal action is at each state in the environment (where each state can include as many variables as needed, much more than just 'where am i'). The trick then is maintain just enough state that helps the bot make good decisions without having a distinct point in your state 'space' for every possible combination of previous actions.
To put that in more concrete terms, if you were to build a chess bot you would probably have trouble if you tried to create a decision policy that made decisions based on all previous moves since the set of all possible combinations of chess moves grows really quickly. Even a simpler model of where every piece is on the board is still a very large state space so you have to find a way to simplify what you keep track of. But notice that you do get to keep track of some state so that your bot doesn't just keep trying to make a left term into a wall over and over again.
The wikipedia article is pretty jargon heavy but this tutorial does a much better job translating the concepts into real world examples.
The one catch is that you do need to be able to define rewards to provide as the positive 'reinforcement'. That is you need to be able to define the states that the bot is trying to get to, otherwise it will just continue forever.
There is precedent: the monstrous rog-o-matic program succeeded in playing rogue and even returned with the amulet of Yendor a few times. Unfortunately, rogue was only released an a binary, not source, so it has died (unless you can set up a 4.3BSD system on a MicroVAX), leaving rog-o-matic unable to play any of the clones. It just hangs cos they're not close enough emulations.
However, rog-o-matic is, I think, my favourite program of all time, not only because of what it achieved but because of the readability of the code and the comprehensible intelligence of its algorithms. It used "genetic inheritance": a new player would inherit a combination of preferences from a previous pair of successful players, with some random offset, then be pitted against the machine. More successful preferences would migrate up in the gene pool and less successful ones down.
The source can be hard to find these days, but searching "rogomatic" will set you on the path.
I doubt bayesian analysis will get you far because most of NetHack is highly contextual. There are very few actions which are always a bad idea; most are also life-savers in the "right" situation (an extreme example is eating a cockatrice: that's bad, unless you are starving and currently polymorphed into a stone-resistant monster, in which case eating the cockatrice is the right thing to do). Some of those "almost bad" actions are required to win the game (e.g. coming up the stairs on level 1, or deliberately falling in traps to reach Gehennom).
What you could try would be trying to do it at the "meta" level. Design the bot as choosing randomly among a variety of "elementary behaviors". Then try to measure how these bots fare. Then extract the combinations of behaviors which seem to promote survival; bayesian analysis could do that among a wide corpus of games along with their "success level". For instance, if there are behaviors "pick up daggers" and "avoid engaging monsters in melee", I would assume that analysis would show that those two behaviors fit well together: bots which pick daggers up without using them, and bots which try to throw missiles at monsters without gathering such missiles, will probably fare worse.
This somehow mimics what learning gamers often ask for in rec.games.roguelike.nethack. Most questions are similar to: "should I drink unknown potions to identify them ?" or "what level should be my character before going that deep in the dungeon ?". Answers to those questions heavily depend on what else the player is doing, and there is no good absolute answer.
A difficult point here is how to measure the success at survival. If you simply try to maximize the time spent before dying, then you will favor bots which never leave the first levels; those may live long but will never win the game. If you measure success by how deep the character goes before dying then the best bots will be archeologists (who start with a pick-axe) in a digging frenzy.
Apparently there are a good number of Nethack bots out there. Check out this listing:
In nethack unknown actions usually have a boolean effect -- either you gain or you loose. Bayesian networks base around "fuzzy logic" values -- an action may give a gain with a given probability. Hence, you don't need a bayesian network, just a list of "discovered effects" and wether they are good or bad.
No need to eat the Cockatrice again, is there?
All in all it depends how much "knowledge" you want to give the bot as starters. Do you want him to learn everything "the hard way", or will you feed him spoilers 'till he's stuffed?
I would like to know if there are any tools that can help me model C applications i.e. Functional programming.
E.g. I'm currently building a shared library.
But to communicate my design visually, I need something like UML. I would like to do this so that the person reviewing my design need not read through 100s of pages of functions, variables and so on.
I have read about UML for C, which I'm considering.
If there is anything better out there, please let me know.
The bottom line is to visualize the design of C applications and modules without reading through 100s of pages of text, because it takes time and is difficult for the reviewers.
Any help in this area from the experts here would be much appreciated.
Thanks.
A well written text documentation brings you a far. Much further than any UML diagram could ever achieve.
You should split this in two parts:
What do you want to say?
What's the best way to saying it?
Whatever formalism you use to answer the second part, you should be sure it's not ambigous.
The good of UML is that a lot of semantic is already defined by the language so you don't have to include a definition of what those boxes, lines and arrows mean in a collaboration diagram.
But most importantly, documenting something means create a path for others to understand the subject you are documenting. A very precise description that offers no clue on how to read it is as good as none. So, use UML, Finite state machines, ER diagrams, plain English, whatever you want but be sure to include a logical path that your "readers" can follow to understand what's going on.
I had a friend that was a fan of "preciseness at all cost" and it would ask us to go through all the details before some sort of meaning would emerge.
I once ask him to do this experiment: on his next trip to an unknown city, he would have to carry the most precise map he could get. Much better, he would have to carry a 1:1 map of the city with every single detail exactly reported in scale. That way he couldn't get lost!
He declined but I would love to see him trying to use that map. Just even folding it! :)
Whatever you like. It's not a standard but many devs use it and understand it. If it does help you to communicate with other people and document your work -> its for you. If it just takes too much time and you think it's not effective, drop it. Also, don't bother with all details, as long as it resembles UML and your team can work with it, it's fine.
It's meant to help you, not waste you time.