Home > Community > Forums > Custom IC SKILL > skill precision for floating points

Email

* Required Fields

Recipients email * (separate multiple addresses with commas)

Your name *

Your email *

Message *

Contact Us

* Required Fields
First Name *

Last Name *

Email *

Company / Institution *

Comments: *

 skill precision for floating points 

Last post Tue, Sep 30 2008 1:55 AM by Galaind. 4 replies.
Started by Galaind 29 Sep 2008 04:51 AM. Topic has 4 replies and 2512 views
Page 1 of 1 (5 items)
Sort Posts:
  • Mon, Sep 29 2008 4:51 AM

    • Galaind
    • Not Ranked
    • Joined on Mon, Sep 29 2008
    • Posts 3
    • Points 45
    skill precision for floating points Reply

    Hi,

    I was wondering about how skill store floating points number,

    Can any one provide more info

    • Post Points: 20
  • Mon, Sep 29 2008 5:09 AM

    • oldmouldy
    • Top 10 Contributor
    • Joined on Tue, Jul 15 2008
    • Woking, Surrey
    • Posts 1,387
    • Points 23,625
    Re: skill precision for floating points Reply

    Refer to page 133 of the Skill Language User Guide, "SKILL stores all floating point numbers in double precision"

    • Post Points: 20
  • Mon, Sep 29 2008 5:18 AM

    • Galaind
    • Not Ranked
    • Joined on Mon, Sep 29 2008
    • Posts 3
    • Points 45
    Re: skill precision for floating points Reply

    Thanks alot for your help,

    But is this mean that skill repesents number as IEEE compliant double precision

    I'm facing some problems in calculating layout points some points after calculation is a single grid off the expceted place.

    • Post Points: 20
  • Mon, Sep 29 2008 10:42 AM

    • Tongju
    • Top 200 Contributor
    • Joined on Mon, Jul 14 2008
    • San Jose, CA
    • Posts 38
    • Points 505
    Re: skill precision for floating points Reply

    When you got your result back onto grid, which function did you use?  A while ago, I used function "int" by mistake, it cut my result by one grid down from time to time.  Then, I switched to anothre function "round" or some formula like int(result/grid + 0.0000001)*grid to work-around it, then, everything looks ok to me.

     Tongju

    • Post Points: 20
  • Tue, Sep 30 2008 1:55 AM

    • Galaind
    • Not Ranked
    • Joined on Mon, Sep 29 2008
    • Posts 3
    • Points 45
    Re: skill precision for floating points Reply

    thanks for your help, I tried round and it passed the correct results to me.

    • Post Points: 5
Page 1 of 1 (5 items)
Sort Posts:
Started by Galaind at 29 Sep 2008 04:51 AM. Topic has 4 replies.