If I have a simple grammar in tree-sitter:
rules: {
expr: $ => choice(
/[0-9]+/,
prec.right(seq($.expr, /[+-]/, $.expr)),
)
}
And an input:
3+4
I get the followng CST:
(start [0, 0] - [0, 3]
(expr [0, 0] - [0, 3]
(expr [0, 0] - [0, 1])
(expr [0, 2] - [0, 3])))
So my question is, how do I get the values, i.e. what was parsed, from these nodes/leafes. I somehow have to evaluate the tree. I'm certainly sure there is way, because I can also do syntax-highlighting with tree-sitter, for what I need the values (I guess). But I read the documentation and couldn't find any note, how to do it.
Tree-sitter's syntax tree doesn't store copies of the input text. So to get the text of a particular token, you would have to use the ranges that Tree-sitter gives you to compute slices of your original source code.
In the python binding, this looks like this:
source_code_bytes = b'3 + 4'
tree = parser.parse(source_code_bytes)
node1 = tree.root_node.children[0].children[0]
node1_text = source_code_bytes[node1.start_byte:node1.end_byte].decode('utf8')
assert node1_text == '3'
In some language bindings, like the wasm binding, there is a .text helper for making this easier.
There is an open issue for adding this kind of helper function to the python binding.
Related
Based on a 1D binary mask, for example, np.array([0,0,0,1,1,1,0,0,1,1,0]), I would like to extract the columns of another array, indicated by the 1's in the binary mask, as as sub-arrays/separate blocks, like [9, 3.5, 7]) and [2.8, 9.1] (I am just making up the numbers to illustrate the point).
So far what I have (again just as a demo to illustrate what my goal is, not the data where this operation will be performed):
arr = torch.from_numpy(np.array([0,0,0,1,1,1,0,0,1,1,0]))
split_idx = torch.where(torch.diff(arr) == 1)[0]+1
torch.tensor_split(arr, split_idx.tolist())
The output is:
(tensor([0, 0, 0]),
tensor([1, 1, 1]),
tensor([0, 0]),
tensor([1, 1]),
tensor([0]))
What I would like to have in the end is:
(tensor([1, 1, 1]),
tensor([1, 1]))
Do you know how to implement it, preferably in pytorch, but numpy functions are also fine. A million thanks in advance!!
You can construct your tensor of slice indices with your approach. Only thing is you were missing the indices for the position of the end of each slice. You can do something like:
>>> slices = arr.diff().abs().nonzero().flatten()+1
tensor([ 3, 6, 8, 10])
Then apply tensor_split and slice to only keep every other element:
>>> torch.tensor_split(arr, slices)[1::2]
(tensor([1, 1, 1]), tensor([1, 1]))
class Solution:
def transformArray(self, arr: List[int]) -> List[int]:
x=arr
while True:
f=True
for i in range(1,len(arr)-1):
if arr[i-1]<arr[i] and arr[i]>arr[i+1]:
f=False
x[i]=x[i]-1
print(x[i])
if arr[i-1]>arr[i] and arr[i]<arr[i+1]:
f=False
x[i]=x[i]+1
print(x[i])
#print(x)
x=arr
if f==True:
break
return x
In the above code both the if statements don't execute , only the second one does. I have tried using elif but it still doesn't work. What am i missing here?
For your code, I considered two types of examples as input for the array list
For example 1, when the arr = [1, 2, 1, 4], the 2nd element is bigger than 1st and 3rd
The first if statement (if arr[i-1]<arr[i] and arr[i]>arr[i+1]:) is working, because both the conditions are met and it gives the output x = [1, 1, 1, 4]
In example 2, when the arr = [3, 2, 3, 4], the 2nd element is smaller than the 1st and 3rd
The second if statement (if arr[i-1]>arr[i] and arr[i]<arr[i+1]:) is working, because both the conditions are met and it gives the output x = [3, 3, 3, 4]
So, the working of if statements largely depends on the elements in the array. Both the if statements' purpose is totally opposite. If one satisfies the condition the other will not.
Hope my answer provides some clarification.
i have multiple feature columns and a result column where i want to predict if something happens or not.
so I'm training my model and finally i do
const predictions = model.predict(xTest).argMax(-1);
this returns a tensor and when getting the data with:
predictions.dataSync ()
i get values like [0, 1, 1, 1, 0, ...]
is there any way to get probabilities like in python? [0.121, 0.421, 0.8621, ...]
I only found one result:
https://groups.google.com/a/tensorflow.org/g/tfjs/c/TvcB69MUj_I?pli=1
is this still the case? are there no probabilities in javascript?
tf.argMax returns the indices of the maximum value along the axis. If you rather want to have the maximum value itself you could use tf.max instead
const x = tf.tensor2d([[1, 2, 3],[ 4, 8, 4]]);
x.max(-1).print() // [3, 8]
x.argMax(-1).print() // [2, 1]
Here is a ruby array:
array = ['x', 3, 0, 4, 4, 7]
I want to map through array, take every integer in the array (except for 0 ) minus 2, and return a new array.
When there are letters, no change will be made to the letters.
Example output:
['x', 1, 0, 2, 2, 5]
This is what I have, but I got the error message saying "undefined method integer? "can someone tell me what is the problem?
def minusNumber(array)
array.map do |e|
if e.integer? && e !== 0
e - 2
end
end
end
The other answers here will work fine with your current input. Something like:
def minusNumber(array)
array.map do |e|
if e.is_a?(Integer) && e != 0
e - 2
else
e
end
end
end
But here is a more flexible solution. This might be a bit too advanced for you where you are now, but all learning is good learning :-)
Ruby is a language that allows polymorphism in its variables. You can see, using the example input for your method, that the variable e may contain a String object or an Integer object. But it actually may contain any type of object, and Ruby would not care one bit, unless it encounters an error in using the variable.
So. In your example, you need to keep Integers in the output. But what if in the future you need to pass in an array containing some Score objects, and you'd need those in your output too? This is a brand new class that you haven't even written yet, but you know you will later on down the line. There's a way you can re-write your method that will anticipate this future class, and all other Integer-type classes you may someday write.
Instead of using #is_a? to check the type of the object, use #respond_to? to check what methods it implements.
Any class that can be used as an integer should implement the #to_int method. Integer certainly does, and your future Score class will, but String does not. And neither does any other class that can't be considered like an integer. So this will work for all types of values, correctly separating those that respond to #to_int and those that don't.
def minusNumber(array)
array.map do |e|
if e.respond_to?(:to_int) && e != 0
e - 2
else
e
end
end
end
Again, this stuff might be a bit advanced, but it's good to get in the habit early of thinking of variables in terms of its methods as opposed to its type. This way of thinking will really help you out later on.
Here's another take on this:
array = ['x', 3, 0, 4, 4, 7]
transformed = array.map do |e|
case e
when 0, String
e
when Integer
e - 2
else
fail 'unexpected input'
end
end
transformed # => ["x", 1, 0, 2, 2, 5]
It's a pity that you have to keep the elements from which you didn't subtract 2. I really wanted to do something like this
array.grep(Integer).reject(&:zero?).map{|i| i - 2 } # => [1, 2, 2, 5]
Couldn't find a way yet (which preserves the unprocessed items).
You can do this without checking for type.
use of kind_of? is a code smell that says your code is procedural, not object oriented…
https://www.sandimetz.com/blog/2009/06/12/ruby-case-statements-and-kind-of
def someMethod(arr)
arr.map do |item|
next item unless item.to_i.nonzero?
item - 2
end
end
someMethod(["x", 3, 0, 4, 4, 7])
> ["x", 1, 0, 2, 2, 5]
Short and sweet.
array.map { |v| v.is_a?(Integer) && !v.zero? ? v - 2 : v }
This eliminates all the needless case statements, multiple iteration, and transformations. Simply takes each element, if it is an integer and greater than 0, then subtracts 2, otherwise does nothing.
As mentioned in the comment above you should use is_a? method to check the datatype of an element. The code below should work as expected:
def minusNumber(array)
array.map do |e|
if e.is_a?(String) || e == 0
e
else
e - 2
end
end
end
Instead of the if statement you could also do:
(e.is_a?(String) || e == 0 ) ? e : e-2
Please note that the original object is unchanged unless you use map!
I think this is legible and elegant:
array.map { |v| v.is_a?(Integer) && v == 0 ? v : v -2 }
Ruby has lots of nice ways of iterating and directly returning that result. This mostly involve array methods. For example:
def ten_times_tables
(1..5).map { |i| i * 10 }
end
ten_times_tables # => [10, 20, 30, 40, 50]
However, I sometimes want to iterate using while and directly return the resulting array. For example, the contents of the array may depend on the expected final value or some accumulator, or even on conditions outside of our control.
A (contrived) example might look like:
def fibonacci_up_to(max_number)
sequence = [1, 1]
while sequence.last < max_number
sequence << sequence[-2..-1].reduce(:+)
end
sequence
end
fibonacci_up_to(5) # => [1, 1, 2, 3, 5]
To me, this sort of approach feels quite "un-Ruby". The fact that I construct, name, and later return an array feels like an anti-pattern. So far, the best I can come up with is using tap, but it still feels quite icky (and quite nested):
def fibonacci_up_to(max_number)
[1, 1].tap do |sequence|
while sequence.last < max_number
sequence << sequence[-2..-1].reduce(:+)
end
end
end
Does anyone else have any cleverer solutions to this sort of problem?
Something you might want to look into for situations like this (though maybe your contrived example fits this a lot better than your actual use case) is creating an Enumerator, so your contrived example becomes:
From the docs for initialize:
fib = Enumerator.new do |y|
a = b = 1
loop do
y << a
a, b = b, a + b
end
end
and then call it:
p fib.take_while { |elem| elem <= 5 }
#=> [1, 1, 2, 3, 5]
So, you create an enumerator which iterates all your values and then once you have that, you can iterate through it and collect the values you want for your array in any of the usual Ruby-ish ways
Similar to Simple Lime's Enumerator solution, you can write a method that wraps itself in an Enumerator:
def fibonacci_up_to(max_number)
return enum_for(__callee__, max_number) unless block_given?
a = b = 1
while a <= max_number
yield a
a, b = b, a + b
end
end
fibonacci_up_to(5).to_a # => [1, 1, 2, 3, 5]
This achieves the same result as returning an Enumerator instance from a method, but it looks a bit nicer and you can use the yield keyword instead of a yielder block variable. It also lets you do neat things like:
fibonacci_up_to(5) do |i|
# ..
end