Here are few True False questions: Somebody please answer these:
Let F(0) = 1, and let F(n) = 2^F(n-1) for n>0.
Then Is F Turing-Computable?
No language which has an ambiguous context-free grammar can be accepted by a DPDA. Is this true ? If not which grammar is that.
For (1), ask yourself this question: could you write a computer program that could compute this function? If so, then by the Church-Turing thesis, then you know that there must be a TM that could do the same computation, so the function is computable. If not, then you know that no TM could evaluate the function either.
For (2), remember that ambiguity is a property of grammars. There can be multiple grammars for the same language.
Hope this helps!
I guess the first one is true.
According to the Church–Turing thesis, computable functions are exactly the functions that can be calculated using a mechanical calculation device given unlimited amounts of time and storage space. Equivalently, this thesis states that any function which has an algorithm is computable.
Related
I just cannot seem to understand the difference. For me it looks like both just go through an expression and apply the chain rule.. What am I missing?
There are 3 popular methods to calculate the derivative:
Numerical differentiation
Symbolic differentiation
Automatic differentiation
Numerical differentiation relies on the definition of the derivative: , where you put a very small h and evaluate function in two places. This is the most basic formula and on practice people use other formulas which give smaller estimation error. This way of calculating a derivative is suitable mostly if you do not know your function and can only sample it. Also it requires a lot of computation for a high-dim function.
Symbolic differentiation manipulates mathematical expressions. If you ever used matlab or mathematica, then you saw something like this
Here for every math expression they know the derivative and use various rules (product rule, chain rule) to calculate the resulting derivative. Then they simplify the end expression to obtain the resulting expression.
Automatic differentiation manipulates blocks of computer programs. A differentiator has the rules for taking the derivative of each element of a program (when you define any op in core TF, you need to register a gradient for this op). It also uses chain rule to break complex expressions into simpler ones. Here is a good example how it works in real TF programs with some explanation.
You might think that Automatic differentiation is the same as Symbolic differentiation (in one place they operate on math expression, in another on computer programs). And yes, they are sometimes very similar. But for control flow statements (`if, while, loops) the results can be very different:
symbolic differentiation leads to inefficient code (unless carefully
done) and faces the difficulty of converting a computer program into a
single expression
It is a common claim, that automatic differentiation and symbolic differentiation are different. However, this is not true. Forward mode automatic differentiation and symbolic differentiation are in fact equivalent. Please see this paper.
In short, they both apply the chain rule from the input variables to the output variables of an expression graph. It is often said, that symbolic differentiation operates on mathematical expressions and automatic differentiation on computer programs. In the end, they are actually both represented as expression graphs.
On the other hand, automatic differentiation also provides more modes. For instance, when applying the chain rule from output variables to input variables then this is called reverse mode automatic differentiation.
"For me it looks like both just go through an expression and apply the chain rule. What am I missing?"
What you're missing is that AD works with numerical values, while symbolic differentiation works with symbols which represent those values. Let's look at simple example to flesh this out.
Suppose I want to compute the derivative of the expression y = x^2.
If I were doing symbolic differentiation, I would start with the symbol x, and I would square it to get y = x^2, and then I would use the chain rule to know that the dervivate dy/dx = 2x. Now, if I want the derivative for x=5, I can plug that into my expression, and get the derivative. But since I have the expression for the derivative, I can plug in any value of x and compute the derivative without having to repeat the chain rule computations.
If I were doing automatic differentiation, I would start with the value x = 5, and then compute y = 5^2 = 25, and compute the derivative as dy/dx = 2*5 = 10. I would have computed the value and the derivative. However, I know nothing about the value of the derivative at x=4. I would have to repeat the process with x=4 to get the derivative at x=4.
For example, I can type into Google or WolframAlpha 6+6, or 2+237, which could be programmed by asking a user for a and b, then evaluating return a+b. However, I might also type 5*5^(e) or any other combination, yet the program is hard-coded to only evaluate a+b expressions.
It's easy to represent the more complex problems in code, on any common language.
return 5*pow(5,Math.E) #pseudocode
But if I can't expect a user's input to be of a given form, then it isn't as simple as
x = Input("enter coefficient")
b = input("enter base")
p = input("enter power")
print(x*pow(b,p))
With this code, I'm locked-in to my program only able to evaluate a problem of the form x*b^p.
How do people write the code to dynamically handle math expressions of any form?
This might not be a question that 'appropriate' for this venue. But I think it's reasonable to ask. At the risk of having my answer voted out of existence along with the question, I'll offer a brief answer.
Legitimate mathematical expressions, from simple to complicated, obey grammatical rules. Although a legal mathematical expression might seem unintelligible, grammatically speaking it will be far less complicated that the grammar needed to understand small bodies of human utterances.
Still, there are levels of 'understanding' built into the products available on the 'net. Google and WolframAlpha are definitely 'high-end'. They attempt to get as close as possible to defining grammars capable of representing human utterance, in effect at least. Nearer the lower end are products such as Sympy which accept much more strictly defined input.
Once the software decides what part of the input is a noun, and what is a verb, so to speak, it proceeds to perform the actions requested.
To understand more you might have to undertake studies of formal language, artificial intelligence, programming and areas I can't imagine.
I created a special-purpose "programming language" that deliberately (by design) cannot evaluate the same piece of code twice (ie. it cannot loop). It essentially is made to describe a flowchart-like process where each element in the flowchart is a conditional that performs a different test on the same set of data (without being able to modify it). Branches can split and merge, but never in a circular fashion, ie. the flowchart cannot loop back onto itself. When arriving at the end of a branch, the current state is returned and the program exits.
When written down, a typical program superficially resembles a program in a purely functional language, except that no form of recursion is allowed and functions can never return anything; the only way to exit a function is to call another function, or to invoke a general exit statement that returns the current state. A similar effect could also be achieved by taking a structured programming language and removing all loop statements, or by taking an "unstructured" programming language and forbidding any goto or jmp statement that goes backwards in the code.
Now my question is: is there a concise and accurate way to describe such a language? I don't have any formal CS background and it is difficult for me to understand articles about automata theory and formal language theory, so I'm a bit at a loss. I know my language is not Turing complete, and through great pain, I managed to assure myself that my language probably can be classified as a "regular language" (ie. a language that can be evaluated by a read-only Turing machine), but is there a more specific term?
Bonus points if the term is intuitively understandable to an audience that is well-versed in general programming concepts but doesn't have a formal CS background. Also bonus points if there is a specific kind of machine or automaton that evaluates such a language. Oh yeah, keep in mind that we're not evaluating a stream of data - every element has (read-only) access to the full set of input data. :)
I believe that your language is sufficiently powerful to encode precisely the star-free languages. This is a subset of that regular languages in which no expression contains a Kleene star. In other words, it's the language of the empty string, the null set, and individual characters that is closed under concatenation and disjunction. This is equivalent to the set of languages accepted by DFAs that don't have any directed cycles in them.
I can attempt a proof of this here given your description of your language, though I'm not sure it will work precisely correctly because I don't have full access to your language. The assumptions I'm making are as follows:
No functions ever return. Once a function is called, it will never return control flow to the caller.
All calls are resolved statically (that is, you can look at the source code and construct a graph of each function and the set of functions it calls). In other words, there aren't any function pointers.
The call graph is acyclic; for any functions A and B, then exactly one of the following holds: A transitively calls B, B transitively calls A, or neither A nor B transitively call one another.
More generally, the control flow graph is acyclic. Once an expression evaluates, it never evaluates again. This allows us to generalize the above so that instead of thinking of functions calling other functions, we can think of the program as a series of statements that all call one another as a DAG.
Your input is a string where each letter is scanned once and only once, and in the order in which it's given (which seems reasonable given the fact that you're trying to model flowcharts).
Given these assumptions, here's a proof that your programs accept a language iff that language is star-free.
To prove that if there's a star-free language, there's a program in your language that accepts it, begin by constructing the minimum-state DFA for that language. Star-free languages are loop-free and scan the input exactly once, and so it should be easy to build a program in your language from the DFA. In particular, given a state s with a set of transitions to other states based on the next symbol of input, you can write a function that
looks at the next character of input and then calls the function encoding the state being transitioned to. Since the DFA has no directed cycles, the function calls have no directed cycles, and so each statement will be executed exactly once. We now have that (∀ R. is a star-free language → ∃ a program in your language that accepts it).
To prove the reverse direction of implication, we essentially reverse this construction and create an ε-NFA with no cycles that corresponds to your program. Doing a subset construction on this NFA to reduce it to a DFA will not introduce any cycles, and so you'll have a star-free language. The construction is as follows: for each statement si in your program, create a state qi with a transition to each of the states corresponding to the other statements in your program that are one hop away from that statement. The transitions to those states will be labeled with the symbols of input consumed making each of the decisions, or ε if the transition occurs without consuming any input. This shows that (∀ programs P in your language, &exists; a star-free language R the accepts just the strings accepted by your language).
Taken together, this shows that your programs have identically the power of the star-free languages.
Of course, the assumptions I made on what your programs can do might be too limited. You might have random-access to the input sequence, which I think can be handled with a modification of the above construction. If you can potentially have cycles in execution, then this whole construction breaks. But, even if I'm wrong, I still had a lot of fun thinking about this, and thank you for an enjoyable evening. :-)
Hope this helps!
I know this question is somewhat old, but for posterity, the phrase you are looking for is "decision tree". See http://en.wikipedia.org/wiki/Decision_tree_model for details. I believe this captures exactly what you have done and has a pretty descriptive name to boot!
these days I have been studying about NP problems, computational complexity and theory. I believe I have finally grasped the concepts of Turing Machine, but I have a couple of doubts.
I can accept that a non-deterministic turing machine has several options of what to do for a given state and symbol being read and that it will always pick the best option, as stated by wikipedia
How does the NTM "know" which of these
actions it should take? There are two
ways of looking at it. One is to say
that the machine is the "luckiest
possible guesser"; it always picks the
transition which eventually leads to
an accepting state, if there is such a
transition. The other is to imagine
that the machine "branches" into many
copies, each of which follows one of
the possible transitions. Whereas a
DTM has a single "computation path"
that it follows, an NTM has a
"computation tree". If any branch of
the tree halts with an "accept"
condition, we say that the NTM accepts
the input.
What I can not understand is, since this is an imaginary machine, what do we gain from saying that it can solve NP problems in polynomial time? I mean, I could also theorize of a magical machine that solves NP problems in O(1), what do I gain from that if it may never exist?
Thanks in advance.
A non-deterministic Turing machine is a tricky concept to grasp. Try some other viewpoints:
Instead of running a magical Turing machine that is the luckiest possible guesser, run an even more magical meta-machine that sets up an infinite number of randomly guessing independent Turing machines in parallel universes. Every possible sequence of guesses is made in some universe. If in at least one of the universes the machine halts and accepts the input, that's enough: the problem instance is accepted by the meta-machine that set up these parallel universes. If in all universes the machine rejects or fails to halt, the meta-machine rejects the instance.
Instead of any kind of guessing or branching, think of one person trying to convince another person that the instance should be accepted. The first person provides the set of choices to be made by the non-deterministic Turing machine, and the second person checks whether the machine accepts the input with those choices. If it does, the second person is convinced; if it does not, the first person has failed (which may be either because the instance cannot be accepted with any sequence of choices, or because the first person chose a poor sequence of choices).
Forget Turing machines. A problem is in NP if it can be described by a formula in existential second-order logic. That is, you take plain-vanilla propositional logic, allow any quantifiers over propositional variables, and allow tacking at the beginning existential quantifiers over sets, relations, and functions. For example, graph three-colorability can be described by a formula that starts with existential quantification over colors (sets of nodes):
∃ R ∃ G ∃ B
Every node must be colored:
∃ R ∃ G ∃ B (∀ x (R(x) ∨ G(x) ∨ B(x)))
and no two adjacent nodes may have the same color – call the edge relation E:
∃ R ∃ G ∃ B (∀ x (R(x) ∨ G(x) ∨ B(x))) ∧ (∀ x,y ¬ (E(x,y) ∧ ((R(x) ∧ R(y)) ∨ (G(x) ∧ G(y)) ∨ (B(x) ∧ B(y)))))
The existential quantification over second-order variables is like a non-deterministic Turing machine making perfect guesses. If you want to convince someone that a formula ∃ X (...) is true, you can start by giving the value of X. That polynomial-time NTMs and these formulas not just "like" but actually equivalent is Fagin's theorem, which started the field of descriptive complexity: complexity classes characterized not by Turing machines but by classes of logical formulas.
You also said
I could also theorize of a magical machine that solves NP problems in O(1)
Yes, you can. These are called oracle machines (no relation to the DBMS) and they have yielded interesting results in complexity theory. For example, the Baker–Gill–Solovay theorem states that there are oracles A and B such that for Turing machines that have access to A, P=NP, but for Turing machines that have access to B, P≠NP. (A is a very powerful oracle that makes non-determinism irrelevant; the definition of B is a bit complicated and involves a diagonalization trick.) This is a kind of a meta-result: any proof solving the P vs NP question must be sensitive enough to the definition of a Turing machine that it fails when you add some kinds of oracles.
The value of non-deterministic Turing machines is that they offer a comparatively simple, computational characterization of the complexity class NP (and others): instead of computation trees or second-order logical formulas, you can think of an almost-ordinary computer that has been (comparatively) slightly modified so that it can make perfect guesses.
What you gain from that is that you can prove that a problem is in NP by proving that it can be solved by an NTM in polynomial time.
In other words you can use NTMs to find out whether a given problem is in NP or not.
By definition, NP stands for nondeterministic polynomial time as can be looked up in Wikipedia.
An incarnation of a nondeterministic Turing machine that randomly chooses and examines (or assembles) the next potential solution will solve an NP problem in polynomial time with some probability (it would solve the problem in poly time with absolute certainty if it were the "luckiest possible guesser").
Therefore, saying that an NTM can solve a problem in polynomial time effectively means that that problem is in NP. This again is equivalent to the definition of the NP class of problems.
I think your answer is in your question. In other words, given a problem you can prove that it is an NP problem if you can find an NTM that solves it.
NP problems are a special class of problems, and the NTM is just a tool to check if the given problem belongs to the class or not.
Note that the NTM is not a specific machine - it is a whole class of machines with well defined rules of what they can and cannot do. In order to use "magical" machines, you need to define them, and show which complexity class of problems they correspond to.
See http://en.wikipedia.org/wiki/Computational_complexity_theory#Complexity_classes
for more info.
From Hebrew Wikipedia - "NTM is mainly a tool for thinking, and it's impossible to actualy implement such machine". You can replace the term "NTM" with "Algorithm that at every step tries all possible steps" or "Algorithm that at every step chooses the best possible next step".. And I think you understand the rest. NTM is here only to help us visualize such algorithm. You can see here how it's supposed to help you visualize (at Pascal Cuoq's answer).
What we gain is that if we have the magical power to guess the correct step, which will always turn out to be correct, we can solve NPC problems in POLYTIME. Of course, we can't always "guess" the correct step. So it's imaginary. But just as imaginary numbers are applicable to real world problems, consequences can be theoretically useful.
One positive aspect of morphing the original problems this way is that we can tackle them from different angles. In a theoretical domain, it is a good thing because we have (1) more approaches we can take (thus more papers) and (2) more tools we can use if they can be phrased in other fields.
The halting problem cannot be solved for Turing complete languages and it can be solved trivially for some non-TC languages like regexes where it always halts.
I was wondering if there are any languages that has both the ability to halt and not halt but admits an algorithm that can determine whether it halts.
The halting problem does not act on languages. Rather, it acts on machines
(i.e., programs): it asks whether a given program halts on a given input.
Perhaps you meant to ask whether it can be solved for other models of
computation (like regular expressions, which you mention, but also like
push-down automata).
Halting can, in general, be detected in models with finite resources (like
regular expressions or, equivalently, finite automata, which have a fixed
number of states and no external storage). This is easily accomplished by
enumerating all possible configurations and checking whether the machine enters
the same configuration twice (indicating an infinite loop); with finite
resources, we can put an upper bound on the amount of time before we must see
a repeated configuration if the machine does not halt.
Usually, models with infinite resources (unbounded TMs and PDAs, for instance),
cannot be halt-checked, but it would be best to investigate the models and
their open problems individually.
(Sorry for all the Wikipedia links, but it actually is a very good resource for
this kind of question.)
Yes. One important class of this kind are primitive recursive functions. This class includes all of the basic things you expect to be able to do with numbers (addition, multiplication, etc.), as well as some complex classes like #adrian has mentioned (regular expressions/finite automata, context-free grammars/pushdown automata). There do, however, exist functions that are not primitive recursive, such as the Ackermann function.
It's actually pretty easy to understand primitive recursive functions. They're the functions that you could get in a programming language that had no true recursion (so a function f cannot call itself, whether directly or by calling another function g that then calls f, etc.) and has no while-loops, instead having bounded for-loops. A bounded for-loop is one like "for i from 1 to r" where r is a variable that has already been computed earlier in the program; also, i cannot be modified within the for-loop. The point of such a programming language is that every program halts.
Most programs we write are actually primitive recursive (I mean, can be translated into such a language).
The short answer is yes, and such languages can even be extremely useful.
There was a discussion about it a few months ago on LtU:
http://lambda-the-ultimate.org/node/2846