In order to solve symbolic planning problems we write action theories.
Popular languages for writing action theories are STRIPS and ADL.
For describing an action we need to provide:
preconditions
effects
For example, in a robot domain, we have Robot and Object classes, and the closeTo and holding properties.
The action pickUp(?robot, ?object) is possible if closeTo(?robot, ?object) holds, and also forall ?o in Object . not holding(?robot, ?o).
How would one represent preconditions with OWL and/or SWRL?
How about action effects?
The Knowrob project suggests that it is possible to use the Qualitative Process Theory (QPT) in combination with OWL-Markup language for implementing actions. A possible precondition would be [1] :
rdf_triple(knowrob:'thermicallyConnectedTo', Dough, HeatSource),
But it was never demonstrated, that this prolog-spaghetti-code will work. OWL is not a real programming language, it's more like a markup language like json. To formalize processes in a deklarative form is a academic quiz but has no relevance for gamecoding or real robotprogramming.
Related
I have two main Questions:
1/ If we speak about OWL 2 semantics in academic manuscripts (ex. thesis) :
do we include the description provided in this W3C official page, which consists of more than one interpretation functions
OR
the one provided in most Description logic and OWL manuscripts? Which consists just of one interpretation function (papers and thesis)???
2/ If we speak about OWL 2 standard reasoning tasks in academic manuscripts (ex. thesis) :
do we speak about object and data properties reasoning tasks( ex. subsumption, satisfiability...) besides those of classes: because most academic manuscripts speak just about classes reasoning tasks in OWL 2;
thank you for telling me which of these alternatives, in both questions, is more correct and formal.
Strictly speaking OWL 2 maps to the DL SROIQ(D) extended with DL safe rules (which provides the semantics for hasKey).
Using one interpretation function is the norm in academic texts.
As AKSW pointed out, standard reasoning tasks are reducible to concept satisfiability (resp. class satisfiability in OWL), hence the reason academic texts tend to refer to concept satisfiability.
Role satisfiability (object/data properties satisfiability) reduces to concept satisfiability by checking satisfiability of the concept $\geq 1 r.\top$. However, there are some limitations when considering object/data properties. See Preventing, Detecting, and Revising Flaws in Object Property Expressions.
The trouble is that “OWL 2 Semantics” is ambiguous: OWL is a well-defined interchange format with several incompatible semantic interpretations. If you like you can refer to that particular document, but it’s important to cite it more specifically as the “OWL 2 Direct Semantics”.
In cases where your work doesn’t involve data types or punning, the SROIQ logic is actually a much simpler and cleaner mathematical formalism...with the caveat that the SROIQ literature is typically written for an academic audience, so this simpler model is usually described in a denser style.
I have some questions related to POMDPs.
What do we mean by controllable actions in a partially observable Markov decision process? Or no controllable actions in hidden Markov states?
When computing policies through value or policy iteration, could we say that the POMDP is an expert system (because we model the environment)? While, when using Q-learning, it is a more flexible system in terms of intelligence or adaptability to a changing environment?
Actions
Controllable actions are the results of choices that the decision maker makes. In the classic POMDP tiger problem, there is a tiger hidden behind one of two doors. At each time step, the decision maker can choose to listen or to open one of the doors. The actions in this scenario are {listen, open left door, open right door}. The transition function from one state to another depends on both the previous state and the action chosen.
In a hidden Markov model (HMM), there are no actions for the decision maker. In the tiger problem context, this means the participant can only listen without opening doors. In this case, the transition function only depends on the previous state, since there are no actions.
For more details on the tiger problem, see Kaelbling Littman and Cassandra's 1998 POMDP paper, Section 5.1. There's also a more introductory walk-through available in this tutorial.
Adaptability
The basic intuition in your question is correct, but can be refined. POMDPs are a class of models, whereas Q-learning is a solution technique. The basic difference in your question is between model-based and model-free approaches. POMDPs are model-based, although the partial observability allows for additional uncertainty. Reinforcement learning can be applied in a model-free context, with Q-learning. The model-free approach will be more flexible for non-stationary problems. That being said, depending on the complexity of the problem, you could incorporate the non-stationarity into the model itself and treat it as an MDP.
There's a very thorough discussion on these non-stationary modelling trade-offs in the answer to this question.
Lastly, it is correct that POMDP's can be considered expert systems. Mazumdar et al (2017) have suggested treating Markov decision processes (MDPs) as expert systems.
I am working on a project using RDF data and I am thinking about implementing a data cleanup method which will run against an RDF triples dataset and flag triples which do not match a certain pattern, based on a custom ontology.
For example, I would like to enforce that class http://myontology/A must denote http://myontology/Busing the predicate http://myontology/denotes. Any instance of Class A which does not denote an instance of Class B should be flagged.
I am wondering if a tool such as the OWLReasoner from OWL-API would have the capability to accomplish something like this, if I designed a custom axiom for the Reasoner. I have reviewed the documentation here: http://owlcs.github.io/owlapi/apidocs_4/org/semanticweb/owlapi/reasoner/OWLReasoner.html
It seems to me that the methods available with the Reasoner might not be up for the purpose which I would like to use them for, but I'm wondering if anyone has experience using OWL-API for this purpose, or knows another tool which could do the trick.
Generally speaking, OWL reasoning is not well suited to finding information that's missing in the input and flagging it up: for example, if you create a class that asserts that an instance of A has exactly one denote relation to an instance of B, and have an instance of A that does not, under Open World assumption the reasoner will just assume that the missing statement is not available, not that you're in violation.
It would be possible to detect incorrect denote uses - if, instead of relating to an instance of B, the relation was to an instance of a class disjoint with B. But this seems a different use case than the one you're after.
You can implement code with the OWL API to do this check, but it likely wouldn't benefit from the ability to reason, and given that you're working at the RDF level I'd think an API like Apache Jena might actually work better for you (you won't need to worry if your input file is not OWL compliant, for example).
I am reading The Pragmatic Programmer: From Journeyman to Master by Andrew Hunt, David Thomas. When I was reading about a term called orthogonality I was thinking that I am getting it right. I was understanding it very well. However, at the end of the chapter a few questions were asked to measure the level of understanding of the subject. While I was trying to answer those questions to myself I realized that I haven't understood it perfectly. So to clarify my understandings I am asking those questions here.
C++ supports multiple inheritance, and Java allows a class to
implement multiple interfaces. What impact does using these facilities
have on orthogonality? Is there a difference in impact between using multiple
inheritance and multiple interfaces?
There are actually three questions bundled up here: (1) What is the impact of supporting multiple inheritance on orthogonality? (2) What is the impact of implementing multiple interfaces on orthogonality? (3) What is the difference between the two sorts of impact?
Firstly, let us get to grips with orthogonality. In The Art of Unix Programming, Eric Raymond explains that "In a purely orthogonal design, operations do not have side effects; each action (whether it's an API call, a macro invocation, or a language operation) changes just one thing without affecting others. There is one and only one way to change each property of whatever system you are controlling."
So, now look at question (1). C++ supports multiple inheritance, so a class in C++ could inherit from two classes that have the same operation but with two different effects. This has the potential to be non-orthogonal, but C++ requires you to state explicitly which parent class has the feature to be invoked. This will limit the operation to only one effect, so orthogonality is maintained. See Multiple inheritance.
And question (2). Java does not allow multiple inheritance. A class can only derive from one base class. Interfaces are used to encode similarities which the classes of various types share, but do not necessarily constitute a class relationship. Java classes can implement multiple interfaces but there is only one class doing the implementation, so there should only be one effect when a method is invoked. Even if a class implements two interfaces which both have a method with the same name and signature, it will implement both methods simultaneously, so there should only be one effect. See Java interface.
And finally question (3). The difference is that C++ and Java maintain orthogonality by different mechanisms: C++ by demanding the the parent is explicitly specified, so there will be no ambiguity in the effect; and Java by implementing similar methods simultaneously so there is only one effect.
Irrespective of any number of interfaces/ classes you extend there will be only one implementation inside that class. Lets say your class is X.
Now orthogonality says - one change should affect only one module.
If you change your implementation of one interface in class X - will it affect other modules/classes using your class X ? Answer is no - because the other modules/classes are coding by interface not implementation.
Hence orthogonality is maintained.
I've been studying hierachial reinforcement learning problems, and while a lot of papers propose interesting ways for learning a policy, they all seem to assume they know in advance a graph structure describing the actions in the domain. For example, The MAXQ Method for Hierarchial Reinforcement Learning by Dietterich describes a complex graph of actions and sub-tasks for a simple Taxi domain, but not how this graph was discovered. How would you learn the hierarchy of this graph, and not just the policy?
In Dietterich's MAXQ, the graph is constructed manually. It's considered to be a task for the system designer, in the same way that coming up with a representation space and reward functions are.
Depending on what you're trying to achieve, you might want to automatically decompose the state space, learn relevant features, or transfer experience from simple tasks to more complex ones.
I'd suggest you just start reading papers that refer to the MAXQ one you linked to. Without knowing what exactly what you want to achieve, I can't be very prescriptive (and I'm not really on top of all the current RL research), but you might find relevant ideas in the work of Luo, Bell & McCollum or the papers by Madden & Howley.
This paper describes one approach that is a good starting point:
N. Mehta, S. Ray, P. Tadepalli, and T. Dietterich. Automatic Discovery and Transfer of MAXQ Hierarchies. In International Conference on Machine Learning, 2008.
http://web.engr.oregonstate.edu/~mehtane/papers/hi-mat.pdf
Say there is this agent out there moving about doing things. You don't know its internal goals (task graph). How do you infer its goals?
In way way, this is impossible. Just as it is impossible for me to know what goal you had mind when you put that box down: maybe you were tired, maybe you saw a killer bee, maybe you had to pee....
You are trying to model an agent's internal goal structure. In order to do that you need some sort of guidance as to what are the set of possible goals and how these are represented by actions. In the research literature this problem has been studied under the terms "plan recognition" and also with the use of POMDP (partially observable markov decision process), but both of these techniques assume you do know something about the other agent's goals.
If you don't know anything about its goals, all you can do is either infer one of the above models (This is what we humans do. I assume others have the same goals I do. I never think, "Oh, he dropped his laptop, he must be ready to lay an egg" cse, he's a human.) or model it as a black box: a simple state-to-actions function then add internal states as needed (hmmmm, someone must have written a paper on this, but I don't know who).