The Android system requires you do draw the full pattern in one session without any "mouse up".
And moving the "mouse" outside the display will result in a "mouse up".

The solution to the four-line problem requires that you continue a full grid position outside the 3x3 grid in both x and y direction. So the phone must layout the 3x3 in a 5x5 to let the user draw outside the box without drawing outside the display.

And no - Android do not care about any "time used". And it is not a "of course" since real users are not always holding the device in a specific way. You may draw with a finger and the phone/pad laying on the desk. You may hold the phone in one hand. You may even have an object - pen, papers or whatever - in the same hand you are drawing with. The user may even have hurt himself and have to use another finger or the other hand. The aim is to know the pattern. Not to try to evaluate acceleration, turns etc. The reason is that the user must always be let in if drawing in the correct order without having to fight with their device.

If you look at the symmetry of the four-line solution, you see that there are very few variations possible where the lines are drawn in a continuous sequence, and where the phone only records the order that the different dots was intersected.

You claim that it matters what starting and ending point. But the four lines do not form a closed loop, so there is just forward and backward multiplied with rotated and mirrored variants.