Poor performance in largish files

[eoghanmurray]

Every keystroke results in a delay of a few seconds in large files (2000+ lines) making the plugin unusable.

Here's a profile trace:

- command-execute                                                1018  97%
 - call-interactively                                            1018  97%
  - funcall-interactively                                        1018  97%
   - eval-last-sexp                                              1018  97%
    - elisp--eval-last-sexp                                      1018  97%
     - elisp--preceding-sexp                                     1018  97%
      - read                                                     1018  97%
       - debug                                                   1018  97%
        - recursive-edit                                         1012  97%
         - command-execute                                       1000  96%
          - call-interactively                                   1000  96%
           - funcall-interactively                                971  93%
            - newline                                             971  93%
             - self-insert-command                                971  93%
              - electric-indent-post-self-insert-function                949  91%
               - indent-according-to-mode                         949  91%
                - php-cautious-indent-line                        949  91%
                 - if                                             949  91%
                  - let                                           949  91%
                   - if                                           949  91%
                    - progn                                       949  91%
                     - funcall                                    949  91%
                      - c-indent-line                             949  91%
                       - c-shift-line-indentation                 949  91%
                        - c-after-change                          943  90%
                         - mapc                                   939  90%
                          - #<compiled 0x157143360ec5>                939  90%
                           - c-after-change-mark-abnormal-strings                939  90%
                            - c-multiline-string-check-final-quote                935  89%
                             - c-literal-limits                   935  89%
                              - c-full-pp-to-literal                935  89%
                               - c-parse-ps-state-below                935  89%
                                - parse-partial-sexp                934  89%
                                 - internal--syntax-propertize                934  89%
                                  - syntax-propertize                934  89%
                                   - php-syntax-propertize-function                915  87%
                                    - while                       914  87%
                                     - if                         913  87%
                                      - php-in-comment-p                913  87%
                                       - nth                      912  87%
                                          syntax-ppss                911  87%
                                    + funcall                       1   0%
                                   + php-syntax-propertize-extend-region                 19   1%
                            + c-literal-limits                      4   0%
                         + c-trim-found-types                       3   0%
                         + c-invalidate-sws-region-after                  1   0%
                        + back-to-indentation                       4   0%
                        + c-before-change                           2   0%
              + c-after-change                                     17   1%
              + #<compiled 0x157143af232d>                          4   0%
              + c-before-change                                     1   0%
           + byte-code                                             29   2%
         + redisplay_internal (C function)                          5   0%
         + timer-event-handler                                      1   0%
+ ...                                                              22   2%

Let me know how I can debug more!

[jlevers]

I'm running into the same issue, except with longer lag and smaller files (running Doom). According to the profiler, most of my CPU time is spent in php-syntax-propertize-function.

[eoghanmurray]

Following through a little more, it may be that it's spending a lot of time trying to syntax highlight areas of code that have been commented out (because they are unused). I got some performance improvement by removing these sections (not a permanent solution as the code is not mine). Is there any way of turning off e.g. php--syntax-propertize-quotes-in-comment?

[phil-s]

Is there any way of turning off e.g. php--syntax-propertize-quotes-in-comment?

Like this?

(advice-add 'php--syntax-propertize-quotes-in-comment :override #'ignore)

[phil-s]

I had cause to revisit this, and the syntax-propertize functionality implemented in php-mode really is a performance killer (and in exchange for relatively slight benefits).

For anyone who finds performance unacceptable, we can suppress this functionality completely via php-mode-hook:

  (setq-local syntax-propertize-function nil)
  (remove-hook 'syntax-propertize-extend-region-functions
               #'php-syntax-propertize-extend-region :local)

Otherwise we can, at minimum, avoid some unnecessary inefficiency in php-syntax-propertize-extend-region:

modified   lisp/php-mode.el
@@ -1042,13 +1042,16 @@ php-syntax-propertize-extend-region
               (let ((maybe (point)))
                 (when (and (re-search-forward (php-heredoc-end-re (match-string 0)) nil t)
                            (> (point) start))
-                  (setq new-start maybe))))
-            (goto-char end)
-            (when (re-search-backward php-heredoc-start-re nil t)
-              (if (re-search-forward (php-heredoc-end-re (match-string 0)) nil t)
+                  (setq new-start maybe)
                   (when (> (point) end)
-                    (setq new-end (point)))
-                (setq new-end (point-max))))
+                    (setq new-end (point))))))
+            (unless new-end
+              (goto-char end)
+              (when (re-search-backward php-heredoc-start-re start t)
+                (if (re-search-forward (php-heredoc-end-re (match-string 0)) nil t)
+                    (when (> (point) end)
+                      (setq new-end (point)))
+                  (setq new-end (point-max)))))
             (when (or new-start new-end)
               (cons (or new-start start) (or new-end end))))
         ;; Cleanup

Experimentally, though, it's really the custom syntax-propertize-function which is the major problem.

This may or may not be on account of the region which is processed regularly being "most of the buffer" in my tests.

(...Some seemingly-incorrect conclusions deleted at this point...)

[phil-s]

I'd reached some wrong conclusions after some apparently-bad testing, so I've deleted part of the previous comment. For the moment I'm suppressing the custom propertization entirely as my workaround.

[phil-s]

I also note that (info "(elisp) Syntax Properties") says of syntax-propertize-function:

     *Caution:* When this variable is non-‘nil’, Emacs removes
     ‘syntax-table’ text properties arbitrarily and relies on
     ‘syntax-propertize-function’ to reapply them.  Thus if this
     facility is used at all, the function must apply *all*
     ‘syntax-table’ text properties used by the major mode.  In
     particular, Modes derived from a CC Mode mode must not use this
     variable, since CC Mode uses other means to apply and remove these
     text properties.

Following that up, I discovered that php-mode no longer derives from c-mode like it used to; but it does still use some CC Mode functionality. I don't know whether or not the above warning is relevant.

[hugot]

@phil-s For reference, how large are the files you are having trouble with? Asking for selfish reasons, I need benchmark material for my package ;)

[phil-s]

The file that caused me to start debugging yesterday is ~9,500 lines (file size is 385Kb).

I do seem to have a compounding problem, as re-testing with a fresh config and just php-mode I'm seeing performance which is significantly better. Still not great in that file, but definitely better than my standard config.

I didn't pick up on obvious other culprits from the profiler, though, so I may need to bisect to try to figure out the difference.

(Edit: It might not be one thing, mind -- plenty of things have reason to call syntax-ppss, so I guess it's not difficult for it to be "fast enough" for php-mode's own purposes, but still a bottle-neck once you add other things into the mix.)

[hugot]

I see. Is it a template with a lot of HTML/text content, or mostly PHP code?

[phil-s]

Entirely PHP code. I'm glad I don't have a 10,000 line template file!

My latest discovery is that I don't immediately suffer the same issue when I edit the file in a fresh Emacs instance using the exact same config which did have issues. This is frustrating, as it means I don't know how to trigger the problem.

This also rings a bell, though -- I realise that I've experienced this same "starts out fine but gets bad later" issue in the past. I don't know whether it's to do with php-mode, or something in CC Mode land, or something else entirely.

[zonuexe]

I just noticed that this topic was revived 3 weeks ago.

It seems that calling syntax-ppss-flush-cache instead of explicit propertyizing causes syntax-propertize--done to have the proper value and the strange behavior goes away. #786

Both issues seem to be affected by the internal state of syntax-ppss, so please try the latest master and provide your feedback.