[Rwp] Plug-in hot spots

Thu Mar 10 05:50:23 EST 2016

I don't know if this is any use at all but I'll throw it out anyway.
Something which was very useful in window-eyes land was a thing we 
called hyperactive windows.
What this is is a frame in jaws speak, an area of the screen or the 
whole screen which looks for a certain condition, text string, color,
what have you and then performs actions when it finds it.
This could be to load another environment of scripted controls
for another part of a plug, or even another plug entirelywhen you can't 
find the window title for what ever reason, it doesn't exist, is equal 
to the handle,
I am drooling down my chin trying to get my head around all the 
permutations of this thing, but hey,
I am learning.
I wonder if some sort of thing like that would be useful?
Seems to me hsc already does some active sensing with that automatic 
title switching thing Jim has going on.
but with the hyperactive windows that concept could be extended to a lot 
more usage.

On 3/9/2016 9:32 PM, James Teh wrote:
> Thanks heaps for your thoughts Jim. I was hoping you'd chime in as the 
> resident expert on such things. :)
>
> Very interesting regarding the unreliability of fractions. I guessed 
> there must have been a reason you didn't promote this as a good 
> option. I guess things might look weird if the whole window just 
> shrank in direct proportion, so it probably doesn't work like that in 
> reality.
>
> It seems many hot spot sets I've seen seem to use the top level window 
> for plug-ins, hence the need for offsets when switching DAWs. You note 
> that you can use a window identified by a particular set of criteria 
> with HSC. The problem is that the real plug-in window usually doesn't 
> have a name and sometimes even the class is useless. Can you do this 
> for a window which is in a certain place in the hierarchy? For 
> example, in REAPER, the plug-in window (when not bridged) is always 
> the last child window of the last child window of the top level 
> window. In Sonar, this is going to be in a different place, but the 
> coordinates should be the same therein. Because the plug-in window is 
> in the same place in the hierarchy for every plug-in, I'm thinking 
> there could be some function in the scripts for that app which allows 
> you to define coordinates relative to "the plug-in window".
>
> High DPI does indeed change things a lot. Interestingly, REAPER isn't 
> DPI aware, so I suspect HSC will see it as if it's 96 DPI. NVDA 
> doesn't because NVDA runs out-of-process, so we see it scaled.
>
> Jamie
>
> On 10/03/2016 1:08 PM, Snowman wrote:
>> A few comments from the HSC side.
>>
>> About the fractional positioning method, I have used this before, 
>> thinking it was a good idea, but kept finding that it did not apply.  
>> I dont' know enough to comment on the general case, world wide. But, 
>> in the applications I worked with, the fraction did not stay 
>> constant, as you resized the window.  That was especially true for  
>> maximize, and Normal states.  For cases where individula borders are 
>> dragged around, who knows how that works.
>> So, I wrote HSC from the viewpoint that what was most likely to stay 
>> constant was the distance to a particular corner.  HSC will let you 
>> decide what corner to use as the reference.
>> But, I think HotSpot developers are not careful enough to look at 
>> things like that, and tend to just settle on what is working, given 
>> the current configuration, and call it good.
>>
>> Similarly, HSC provides several different ways to determine which 
>> window is used as the reference.  top level window, The application 
>> main window, the window that contains the control with focus, a 
>> window identified by a particular set of criteria like name, control 
>> id, class etc.  Such windows do not need to be part of the 
>> application family tree.  And again, most developers I have worked 
>> with are not terribly aggressive on kicking the tires if the solution 
>> happens to work.  The fact is that, for a given configuraition, any 
>> number of those approaches will work. Question is, which one keeps 
>> working. So, while HSC gives you the tools, you really need to dig 
>> in, and see how things change in order to find the best approach. 
>> Often, that is purely a question of development time, and developers 
>> are eager to move on, since there are usually tons of individual 
>> controls that need to be identified.  So, they just get in a hurry, 
>> and take the expedient approach.
>> Application main window, seems to be a very commonly used method, 
>> which is the worst for pluggIns.
>>
>> I have been told that some PluggIn windows are actually rendered 
>> slightly differently invarious hosts, leading to such things as 
>> slight offsets.  I dont' know the root cause of that. Conceivably, 
>> that is a jaws quirk, who knows which versions. But, HSC gives you  
>> the ability to shift hotSpots by a fixed offset, x and y.  That tool 
>> should allow one to correct for such cases. But, what I need to do is 
>> actually take on the task of adapting a set some time to see how well 
>> that actually works out.  But, it's not for the faint of heart, and a 
>> far less manageable task for one who is not intimately familiar with 
>> the hotspot set.
>>
>> As for screen resolution, I have been struggling for some time to 
>> really understand how that works.  In my experience, using standard 
>> 96DPI, screen resolution doesn't seem to matter.  I know that sounds 
>> surprising.  But, I never had any method that routed to a set of x y 
>> coordinates fail as I changed resolution.  It seemed like the scaling 
>> JAWS was dealing with stayed fixed relative to the applicaition.  
>> What changed, as you modified the resolution,  is where that point 
>> actually turned up on the physical screen.
>> However, I am told that, with newer high-DPI settings machines, that 
>> is no longer the case.  But, I dont' have any experience with that.
>> It may well be that hotSpots defined on a 96DPI machine would not 
>> work on a high resolution laptop, for example.  And, it seems to 
>> depend on the application itself being written in a DPI-aware 
>> fashion.  But, I'm engaging in uninformed speculation.
>>
>>
>>
>>
>>
>>
>>
>> ----- Original Message ----- From: "James Teh" <jamie at nvaccess.org>
>> To: "Reapers Without Peepers mailing list" <rwp at bluegrasspals.com>
>> Sent: Tuesday, March 08, 2016 10:49 PM
>> Subject: [Rwp] Plug-in hot spots
>>
>>
>>> Hi plug-in hot spot creators (HSC, AutoHotkey, whatever),
>>>
>>> As I've watched discussion about creation of hot spots for plug-ins 
>>> and associated problems, I've been starting to wonder about why they 
>>> aren't done in a way which makes them more reliable across different 
>>> configurations. I don't have much experience in this area, so there 
>>> may well be a good reason and I'm just not seeing it. So, I'd 
>>> appreciate some thoughts on the below.
>>>
>>> In the following discussion, let's assume we're dealing with simple 
>>> hot spots; i.e. spots which don't change such as the program browser 
>>> button in Dimension Pro, the load button in SFZ, etc. I realise 
>>> there are many cases where this isn't the case, but that's beyond 
>>> the scope of what I'm getting at here.
>>>
>>> As I understand it, there are two main problems that cause basic hot 
>>> spots to be unreliable:
>>>
>>> 1. Different screen resolutions/DPI settings. If hot spots are 
>>> created for one screen configuration, they sometimes don't work on 
>>> other configurations.
>>> I notice that even though some hot spots use relative coordinates, 
>>> they're still absolute as far as the window is concerned. That is, 
>>> they don't account for the window being a different size, only the 
>>> window being in a different starting position. So, why not use width 
>>> and height fractions? So, if the coordinate is half way across the 
>>> width of the window, you would store 0.5 instead of an absolute 
>>> value like 200. This way, if the window shrinks to half size, you 
>>> can still calculate the correct coordinates. In practice, some 
>>> windows probably don't resize everything in direct proportion, but I 
>>> would have thought this would be more rare than not.
>>>
>>> 2. Different applications. Hot spots created for Sonar don't work in 
>>> REAPER and vice versa. Even in REAPER, hot spots created for bridged 
>>> don't work in non-bridged.
>>> From what I've seen, even hot spots that are relative are calculated 
>>> relative to the top level window. The problem is that the top level 
>>> window is part of the host (e.g. REAPER), not the plug-in. That 
>>> means it's going to be different for every host. So, why not 
>>> calculate the coordinates relative to the window for the plug-in 
>>> itself? The plug-in should render exactly the same way no matter 
>>> where it's hosted. Therefore, if coordinates are all relative to the 
>>> plug-in window, they will work the same regardless of the host used.
>>>
>>> On a related note, I'd be interested to know whether it'd be worth 
>>> implementing some kind of basic hot spot support into OSARA. The 
>>> question is what kind of complexity is needed for the majority of 
>>> plug-ins. I know HSC has support for searching for colours, etc., 
>>> but I never quite grasped how this was applied in practice.
>>>
>>> I'd appreciate any thoughts.
>>>
>>> Jamie
>>>
>>> -- 
>>> James Teh
>>> Executive Director, NV Access Limited
>>> Ph +61 7 3149 3306
>>> www.nvaccess.org
>>> Facebook: http://www.facebook.com/NVAccess
>>> Twitter: @NVAccess
>>> SIP: jamie at nvaccess.org
>>>
>>> _______________________________________________
>>> RWP mailing list
>>> RWP at bluegrasspals.com
>>> http://bluegrasspals.com/mailman/listinfo/rwp
>>>
>>
>>
>> _______________________________________________
>> RWP mailing list
>> RWP at bluegrasspals.com
>> http://bluegrasspals.com/mailman/listinfo/rwp
>