Detecting mouse click on specific SKPath using skiasharp SKControl - winforms

Question

I am using the Skiasharp Windows Forms SKControl to handle drawing actions within my app. Part of the functionality is to be able to click on the canvas to select a specific "layer" and allow the user to apply a different color to each layer. See the image below for an example.
The three layers in this image are drawn as three separate paths. For the most part, the code that I have is working great, except for when those "layers" overlap. If I click on the lightest layer in the above image at a point that is lower than the "point" of the darkest layer, it will select the darker layer.
This makes sense since I am using path.GetBounds() which returns an SKRect. On mouse click I am creating an SKRect from the click location and then calling rect.IntersectsWith(mouseRect) to determine which "layer" I clicked on.
How can I validate layer clicked given the paths are not simple rectangles?
Here is the code that I am using for the "HitTest" (non-optimized since I was debugging)
SKRect hitRect = SKRect.Create(scaledPixelPosition, new SKSize(20f, 20f));
ElementLayerInfo elementLayerInfo = new ElementLayerInfo();
foreach(Element element in _elements)
{
foreach (KeyValuePair<int, ElementLayerInfo> kvp in element.ElementLayers)
{
if (kvp.Value.LayerRect.IntersectsWith(hitRect))
{
elementLayerInfo = kvp.Value;
}
}
}