Skip to content

支持设置匹配规则强行中断并报错 #391

Closed

Description

@WslzGmzs

最近在使用一些自部署开源模型的时候,会遇到神奇的问题,生成速度骤然降低到个位数,取消重试就恢复正常,希望能够支持运行时的规则,比如匹配到tps过低就直接强行中断并且报错,让客户端自己重试

Activity

  1. changed the title [-]支持设置匹配规则强行中断报错[/-] [+]支持设置匹配规则强行中断并报错[/+] on Oct 30, 2026
  2. WslzGmzs commented on Oct 31, 2026

    @WslzGmzs
    Author

    为这个匹配模型会好点,一个上游不同模型的tps判定可能不同

    @fylorn

  3. fylorn commented on Nov 2, 2026

    @fylorn
    Contributor

    This issue primarily stems from the limitations of your GPU memory. Generally speaking, this is an upstream matter; incorporating speed-detection features throughout the software would make it overly bloated, so there are no plans to implement this for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions