Enhancement request: Proper RTL movement

[yousufinternet]

#Issue Type

• Enhancement request

Emacs version: GNU Emacs 25.2.1
Operating System: Arch Linux with the latest updates
Evil version: 1.2.12
Evil installation type: Manual
Graphical/Terminal: Graphical
Tested in a clean environment: Yes

Reproduction steps

• Start Emacs
• Type some Arabic (Farsi, Hebrew or Urdu) text so that it appears from the right to the left, for example:

هذا نص عربي للتجربة

• Try moving (using right and left arrow keys, or 'h' and 'l' keys) while in normal mode.

Expected behavior

The movement keys should move the cursor to the left when pressing the left arrow key, and to the right when pressing the right arrow key, while on a RTL text.

Actual behavior

The keys move the cursor to opposite of the direction they are supposed to move the cursor in while in a RTL text.

Further notes

Please note that I have only faced this problem when trying to move by characters, other movements that depend on text objects (i.e. pressing 'e' moves correctly to the end of the word on the left side of the cursor) behave correctly.

I hope that there is a quick workaround that makes evil recognise different language environments and reverse its behaviour accordingly.

Thanks and Best Regards

[wasamasa]

You mention that this happens both when using hjkl and arrow keys. This is a clear indicator that your issue is not Evil-related as Evil doesn't cover the arrow keys. As I can reproduce the same problem in emacs -Q with the arrow keys, I'm closing this. Feel free to M-x report-emacs-bug and convince Eli Zaretskii (the person who implemented bidi support) why the current behavior is wrong.

[yousufinternet]

That is strange because I can't reproduce the issue on emacs -Q.
if I run emacs -Q, open an empty file (C-x C-f testfile), and type some RTL text, and move using the arrow keys it moves as it is supposed to move, so I am not sure how you were able to reproduce the issue, but my guess is that you used the lisp evaluation buffer to write the text, which does not support bidi text.

Please note that the problem only occurs in evil normal mode, and not insert mode, that is why I thought that it is evil related, and I am now using emacs without evil, and it's behaving correctly with bidi text.

Thanks for the comment and the suggestion.

[wasamasa]

Indeed, I tried it in *scratch* only because you didn't mention using a specific buffer in the repro. In a text mode buffer I can repro a difference between cursor keys in normal and Emacs state which means that Evil is using its own commands for cursor keys after all.

In vanilla Emacs, C-f and <right> are bound to two different commands, one that isn't bidi-aware and one that is. In Evil, l and <right> are bound to the same bidi-unaware command. The question here is how to deal with this, for that I'd try observing how Vim does things and mirror it if it makes sense, otherwise I'd steal Emacs behavior.

[yousufinternet]

My mistake, I should have clarified that.
As far as I know vim/nvim bidi support is very poor, if any at all, since that was one of the reasons that got me to switch to emacs, any way I am no expert, so I will be awaiting your comments :-)

[wasamasa]

From a quick test with :set arabic in Vim, both hjkl and arrow keys move in the logical direction (as in, the direction the key points to) whereas word movement goes in text direction (as in the direction the text flows). This is opposed to vanilla Emacs where C-f moves in text direction and the arrow keys in logical direction. I'm not sure which of the two to prefer here as I haven't ever authored RTL texts or localized websites for RTL scripts. I can see the point in supporting both styles like vanilla Emacs does, but also in reducing confusion by making both follow logical direction.

[yousufinternet]

Cool, I don't know about your plans about how to resolve this, but I was hopping at least that the desired behaviour for hjkl and arrow keys could be set by some option in future updates to be moving the cursor in the logical direction or in the text flow direction as the user desires.

I have tried vim with :set arabic, and it has the same behaviour in normal and insert mode evil has in normal mode (but not insert mode), the arrow keys and hjkl move to the opposite direction of the text flow as you said, while text object movements move normally in the text flow direction.

[yousufinternet]

Adding this to my .emacs gives me the behaviour I want:

(define-key evil-normal-state-map "h" 'left-char)
(define-key evil-normal-state-map "l" 'right-char)
(define-key evil-normal-state-map [left] 'left-char)
(define-key evil-normal-state-map [right] 'right-char)
(define-key evil-visual-state-map "h" 'left-char)
(define-key evil-visual-state-map "l" 'right-char)
(define-key evil-visual-state-map [left] 'left-char)
(define-key evil-visual-state-map [right] 'right-char)

as you said, the commands forward-char and backward-char are not bidi-aware.. your comments helped me find the correct way to think about his issue, thanks.

[wasamasa]

The above workaround could be made a bit shorter by unbinding the keys, so that Emacs falls back to the (non-Evil) defaults:

(define-key evil-motion-state-map "h" nil)
(define-key evil-motion-state-map "l" nil)
(define-key evil-motion-state-map [left] nil)
(define-key evil-motion-state-map [right] nil)

I'd expect it to break something (like repetition or operator usage), but from short testing nothing seems to be affected.

[yousufinternet]

Adding this to my .emacs gives me the behaviour I want:

(define-key evil-normal-state-map "h" 'left-char)
(define-key evil-normal-state-map "l" 'right-char)
(define-key evil-normal-state-map [left] 'left-char)
(define-key evil-normal-state-map [right] 'right-char)
(define-key evil-visual-state-map "h" 'left-char)
(define-key evil-visual-state-map "l" 'right-char)
(define-key evil-visual-state-map [left] 'left-char)
(define-key evil-visual-state-map [right] 'right-char)

as you said, the commands forward-char and backward-char are not bidi-aware.. your comments helped me find the correct way to think about his issue, thanks.

I had to add:
(setq visual-order-cursor-movement t)

To get this working again after recent updates