[aspiers]

Hi there, I just got a new Google Pixel Slate and Lenovo Active Pen 2 stylus to be my main MSP device.  While the stylus works great with all the drawing apps I've tried, it's really quite laggy when I'm using it to add annotations within MSP.  It's not so slow as to be totally unusable, but the lag is noticeably slower than the other apps on the same device, and also slower than MSP on my old Samsung Galaxy Note 2014.  There is no deterioration in accuracy, but the lag appears to be more noticeable when drawing a longer shape (i.e. not lifting the stylus off the screen).  It feels to me like MSP is receiving the input events on time but is struggling to keep up with displaying them on the screen.

Is there any way of fixing this?  As a developer I'd be very willing to help debug it if there is anything I can do to help.  Thanks!

[Zubersoft]

If you tap the circle at the top left corner of the annotations editor, is "Draw highlights behind content" enabled? If so, that can cause considerable slowdown during rendering as it has to perform some of the rendering without hardware acceleration. Let me know if you want the reason behind that, but it's pretty involved and has to do with pixel blending and how the hardware accelerated graphics work in the Android rendering framework. One thing I can do to speed up the rendering (which I did for e-ink devices and older Android devices), is to only draw the highlights behind the page content when the user isn't actively annotating (meaning a finger or stylus isn't touching the screen). That may be why your Samsung Galaxy Note 2014 doesn't have the issue.

If that isn't the cause of the slowdown, let me know. That would indicate something else is going on.

Thanks,
Mike

[aspiers]

Thanks a lot for the quick reply!  Nope - I checked and it wasn't enabled.  Out of curiosity I enabled it and it didn't appear to make any difference.

[Zubersoft]

Do you get the same slowdown when you use a finger to annotate vs the stylus? I'm trying to determine if the stylus has any impact on this problem or if it's related to how MobileSheetsPro stores and renders annotations.

Thanks,
Mike

[aspiers]

Hrm, I just checked and actually I suspect it's the same with fingers.  Sorry if I initially led you down the wrong path.  So maybe the issue is specific to ChromeOS devices, or even the Pixel Slate in particular.  Wild guess - could it perhaps be related to the fact that the device is very high resolution?  It's 3000x2000, so presumably that means a lot more pixels to track per annotation line than most devices?

[Zubersoft]

I do my testing on a Samsung Tab S4 which has a resolution of 2560x1600, so it's not that different from the Pixel Slate. I experience no slowdown at all while annotating as far as I can tell. So this leads me to believe the problem may have something to do with ChromeOS. I have a Samsung Chromebook Plus I can perform some tests on, but the specs are not nearly as good as the Pixel Slate. If you can capture a video showing the slowdown, this would help me to understand how significant the problem is compared to other apps on your device. I just want to be able to quantify the slowdown (i.e. 20% slower vs 50% slower) to see if perhaps some smaller optimizations could be made to help with this versus requiring a completely different rendering model entirely. Right now, every time you draw something, all annotations on the screen are constantly redrawn in Android's rendering framework as they are all owned by the same parent view (the page behind is not redrawn though, reducing the required rendering load). If you get the slowdown even when annotating a page with no previous annotations on it while drawing a circle in freeform mode, let's say, then that would indicate that the way I'm rendering the freeform annotation is an issue.  If the slowdown only occurs on pages where you already have a large number of annotations, then that points to a problem with how much is having to be redrawn over and over. In that case, I could look into an optimization where the only thing being rendered while the stylus or finger is down is the current drawing, which would be drawn into a separate layer. Once the drawing is complete, I would commit it to the same layer as the other annotations. 

The more information you can provide, the better. Without being able to see the problem myself, it will be hard to know what to change. I'll set up my Chromebook here in a bit and run through some tests on that at least.

Thanks,
Mike

[aspiers]

Sure thing, I can submit a video soon!  Thanks!

[otnt]

I've tried Pixel Slate with Google's Pixelbook Pen and I can confirm slight lag but as MS is not a drawing app it is not a concern for me at all.
It's still perfectly fine to draw freeform bowings, fingerings etc.
Nothing to worry about.

[aspiers]

OK here's a video: https://youtu.be/G1VdXHteCww

It's certainly tolerable, but it would be awesome if it was fixed since the comparison with other apps on the same device clearly shows the issue is not purely with the hardware.

[Zubersoft]

This one is going to take some work to figure out. What my code is doing is fairly simple in nature, so I'm going to need to do some profiling to see if I can figure out if there are any bottlenecks in my touch processing and rendering code. If there aren't, then my approach to drawing is too slow. I'm using Google's Path class to just create quads between each touch point so that it's smooth, and in the rendering part of the code, I call a simple "canvas.drawPath" method that draws whatever is stored in the Path object. Apparently every time you update a path though, it does some pretty heavy calculations and processing behind the scenes though, so perhaps I'm actually processing the touch actions too quickly. Perhaps what I need to do is cache the touch actions for a small amount of time and only update the path and redraw every 25-50ms. I'll see what I can figure out.

On a side note, which model of the Google Pixel Slate do you have Aspiers (8th gen intel core m3/i5/i7)?

Thanks,
Mike

Update:
I just noticed this detail in the Squid store listing:

We partnered with Google to bring incredibly fast, low-latency ink to Squid on the Google Pixelbook and Samsung Chromebook Plus (with Chrome 62). See the difference: https://youtu.be/Oc66T_e7xwU

It appears they utilized a low-latency ink api that is available for ChromeOS. I've been searching for this, but I can't find it anywhere on the web. Google doesn't seem to have a way for me to contact them directly about this so I'm not sure what the best way is to handle this. If anyone else can find the API, please let me know. Have you tested other drawing apps other than Squid that don't utilize this low-latency ink api? I imagine they will behave the same as MobileSheetsPro. 

I just tested Squid on my Samsung Tab S4, and once the line that I'm drawing gets fairly long, Squid actually starts lagging far worse than MobileSheetsPro. I was drawing a continuous line all over the screen in MobileSheetsPro and the line still kept up with the stylus. With Squid, it started lagging behind pretty badly. So on Android devices, my app is drawing faster, but on ChromeOS, Squid is better due to the low-latency API.

Update #2:
As far as I can tell, it appears you have to utilize the web API for ChromeOS in order to get the low-latency stylus events. That would mean either having to do a rewrite of the rendering and touch input using a WebView embedded in the app (which would be insane), or having to rewrite the whole application as a web-app for ChromeOS. I'm not sure what approach Squid took. Either way, Google does not seem to make it easy for an Android developer to take advantage of these optimizations, so unless someone can point me to something that says differently, I don't believe there is any easy way for me to get low-latency stylus input like Squid and other similar apps.

[otnt]

I would not compare MSPro with handwritten note taking app.
Even Google's Keep app which has handwritten/drawing section is lagging on Pixel Slate m3.
MSPro is superior for a different reasons.
Why would anyone make fast and whole screen scribbles on a sheet music shown in video made by aspiers?