NOTE: Using PyAutoGui library
I am trying to make python click on each icon on screen in order, I have gotten it to successfully print each item on screen with it's box values of left and top in place of X and Y coordinates. But I can't figure out how to get left/top converted into X/Y values for use with the pyautogui.click()
Code:
import pyautogui
coordinates = pyautogui.locateAllOnScreen('eachIcon.png')
for element in coordinates:
print(element)
Prints:
Box(left=124, top=699, width=14, height=14)
What command would I use to extract Left and Top as X and Y number coordinates?
I am entirely new to python and practically new to coding (took a beginners class of C++ in college). I've spent a good hour googling and trying every term I can think of, I'm stumped: hence the post.
To do this you will need to unpack a tuple:
Box has 4 objects Box(left=124, top=699, width=14, height=14) to unpack Box we can just add 4 dummy variables (since we only need x,y) to your for code
for x, y, _, _ in coordinates: or for x, y, w, h in coordinates if you also want the weight and height
I inserted the dummy variables because otherwise we will get a ValueError.
The error occurs when the number of variables doesn't match the number of values so if we had for x, y in coordinates: we would get a ValueError.
You could also access a tuple by doing
x = [0]
y = [1]
But i suggest you to use the for code since you also want to click them in order
coordinates.left coordinates.right etc should do the trick
Related
i need help understanding this code below. the code comes up in a lot of deep learning contexts (especially transformers) where we need to calculate relative distances on a regular 2d grid. for example, in below image (from crossformer paper), distance between points i and j are given as (-1, -2). signs may be confusing, but assume if x_i<x_j, distances will be negative and vice versa. here's the code, i vaguely understand what's happening, but i need to actually fully understand because i have to reapply this logic to other papers with different details - and don't want to keep brute forcing until i get it right.
grid_x, grid_y = torch.meshgrid(h_range, w_range, indexing = 'ij')
grid = torch.stack((grid_x, grid_y))
grid = einops.rearrange(grid, 'c h w -> c (h w)')
grid = (grid[:, :, None] - grid[:, None, :])
it's easy to see grid (on line 2) is a 2 x H x W array (or W X H). 3rd line is confusing to me because i don't get why we flatten. then, we use broadcasting so that 2 x (HW) x 1 - 2 X 1 X HW results in 2 X HW X HW. ok, there are HW points on this grid. each point has a unique distance to each other point (to itself too), hence HW X HW. BUT, i don't understand how to think like this before i get the solution. what's the right way to start thinking about this kind of matrix manipulation problem?
thank you!
enter image description here
I am trying to determine the nearest x value of the user's mouse position on a simple line chart. Is there any way to do this?
I have already tried adding a pointMouseOver and pointsHover listener to the chart instance, and, while they emit some kind of index, the number of indices far exceeds the number of x values in the chart (My chart had around 600 x values, the largest index emitted is around 1000).
this.chart.listen('pointMouseOver', function (event) {
console.log(event.iterator.getIndex())
console.log(event)
})
returns some arbitrary index instead of the nearest x value.
Managed to answer my question myself while looking through the charts API reference - to get the nearest x value on pointsHover, call event.point.get('x'). https://playground.anychart.com/api/_samples/anychart.enums.EventType.pointsHover
I'm using ffmpeg to extract motion vectors using the example doc extract_mvs.c. Issue is that this code only seems to give a few pieces of information: frame number, size of macroblock, source (future or past), source (x and y), and destination (x and y).
Unfortunately, this doesn't say which frame the source comes from in the past or the future (it could come from both, from several past, or several in the future). It also doesn't say what the macroblock type is (which tells similarly useful info). For example, if Source (x and y) equals Destination (x and y) it is impossible to tell if that information is the same as the last frame or if it entered completely new information.
See lines 60-63 in the extract_mvs.c code in ffmpeg
Final question is that for MP4, motion vectors generally have quarter pixel resolution and the resolution given here is clearly rounded to the closest integer. How am I supposed to extract the "true" motion vector information before rounding?
The source (future or past) is based on a relative frame reference given by the direction parameter to add_mb(), but I'm not sure what to make of the logic:
mb->source = direction ? 1 : -1;
In libavutil/motion_vector.h there's a comment, XXX: set exact relative ref frame reference instead of a +/- 1 "direction", so it looks like a known TODO not addressed by the creator of the patch. The value of direction comes from ff_print_debug_info2() where add_mb() is called.
As for the quarter pixels, I think this is also in ff_print_debug_info2() but I don't know enough about motion_val to say what it means:
const int shift = 1 + quarter_sample;
...
int mx = (motion_val[direction][xy][0]>>shift) + sx;
int my = (motion_val[direction][xy][1]>>shift) + sy;
The initial commit shows all the major pieces of this motion vector code. Hopefully this gets you going in the right direction (no pun intended).
I am programming a basic GUI in MATLAB that utilizes the mapping toolbox. The GUI will display a grayscale image and then plot discrete points over the data, all of this over the necessary map projection. It is important that I plot onto map axes (those created by the axesm command) rather than the vanilla cartesian space. I have no problem doing all this from the command line, but I cannot find a way to implement a GUI version and its driving me nuts.
The problem is that I need to specify the map axes as being the child of the parent figure. The normal axes has a property that can be set, doing something like:
axesHandle = axes('Parent', parentHandle, ...);
or
set(axesHandle, 'Parent', parentHandle);
However, there is no equivalent parent property for the map axes created by the axesm function, so I have no way to manipulate the axes within the figure. How can I do this?
Update: If I create a plot within the map axes in an empty figure, get(figureHandle, 'Children') returns the handle of the axesm object (thanks #slayton!), so the map axes object must be implicitly added to the children of the figure by MATLAB.
Should I be concerned that the map axes do not refer back to the parent figure, or should I just let it be? I wonder if this is a classic case of MATLAB forcing me to not comply with the standards the manual tells me to implement.
From reading your question what I think you are trying to do is grab the handle of the axes object. This can be done as the axes is created using either axes or subplot
a = axes();
a = subplot(x,y,z);
% both return an handle to the newly created axes object
Additionally if the axes is created automagically by a function call like plot or image you can get the axes handle that too:
p = plot(1:10); %returns a handle to a line object
a = get(p,'Parent');
i = image(); %returns a handle to an image object
a = get(i, 'Parent');
Finally, neither of those two options is available you can always get the axes handle from its containing figure with:
a = get(figureHandle, 'Children');
Remember though that this will return a vector of axes handles if your figure contains more than one axes.
Finally when it comes time to draw draw your points to the axes that contains your map image you simply need to call:
line(xPoints, yPoints, 'linestyle', 'none', 'marker', '.', 'color', 'r', 'size', 15)
This will draw the vertices of the line using large red dots.
I'm not sure if this answers your question because the code you provided doesn't line up with the question you asked.
The code you provided looks like you are trying to move an axes from one figure to another. You can totally do this!
f = figure('Position', [100 100 100 100]);
a = axes('Parent', f);
pause
f2 = figure('Position', [250 100 100 100]);
set(a,'Parent', f2);
After much trial and error and reading of documentation, I have found that there is no way to explicitly specify the parent of the map axes. Instead, they are implicitly added on top of the current axes. In the instance that no axes exist in the current figure, calling axesm creates an axes object and then places the axesm object inside. When you take this route, you have to grab the axes object handle by calling gca:
mapAxesHandle = axesm(...);
axesHandle = gca(...);
This makes it frustrating to use the mapping toolbox when writing a GUI from scratch, but that's the way Mathworks makes it happen. Thanks to #slayton for useful info. I'd upvote but my reputation is <15 :(
If you have the full relative-3D values of two images looking at the same scene (relative x,y,z), along with the extrinsic/intrinsic parameters between them, how do you project the points from one scene into the other scene, in opencv?
You can't do that in general. There is an infinite number of 3D points (a line in 3d) that get mapped to one point in image space, in the other image this line won't get mapped to a single point, but a line (see the wikipedia article on epipolar geometry). You can compute the line that the point has to be on with the fundamental matrix.
If you do have a depth map, reproject the point into 3D - using the equations on the top of the opencv page on camera calibration, especially this one (it's the only one you need):
u and v are your pixel coordinates, the first matrix is your camera matrix (for the image you are looking at currently), the second one is the matrix containing the extrinsic parameters, Z you know (from your depth map), X and Y are the ones you are looking for - you can solve for those parameters, and then use the same equation to project the point into your other camera. You can probably use the PerspectiveTransform function from opencv to do the work for you, however I can't tell you from the top of my head how to build the projection matrix.
Let the extrinsic parameters be R and t such that camera 1 is [I|0] and camera 2 is [R|t]. So all you have to do is rotate and the translate point cloud 1 with R and t to have it in the same coordinate system as point cloud 2.
Let the two cameras have projection matrices
P1 = K1 [ I | 0]
P2 = K2 [ R | t]
and let the depth of a given point x1 (homogeneous pixel coordinates) on the first camera be Z, the mapping to the second camera is
x2 = K2*R*inverse(K1)*x1 + K2*t/Z
There is no OpenCV function to do this. If the relative motion of the cameras is purely rotational, the mapping becomes a homography so you can use the PerspectiveTransform function.
( Ki = [fxi 0 cxi; 0 fyi cyi; 0 0 1] )