Skip to content

Incorrect format with ExponentialDatumFormatter at resolution limits #165

@jbfaden

Description

@jbfaden

Masafumi pointed out the issue when zooming in on an axis. This Jython code shows the root of the problem:

from org.das2.datum.format import ExponentialDatumFormatter
format= ExponentialDatumFormatter(20,7)

print format.format(datum(4.28230E7),Units.dimensionless)  # 0.0922337203685477500E7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions