- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - OpenVMS
- >
- Simple multiplication gives incorrect results
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-19-2007 05:44 AM
11-19-2007 05:44 AM
Actual example:
FS = f$getdvi("DISK1","FREEBLOCKS")
MB = FS*512
display FS --> 4995264
display MB --> -1737392128
expecting MB to be 2557575168
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-19-2007 06:01 AM
11-19-2007 06:01 AM
Solution1737392128
+2557575168
-----------
4294967296 = hex 100000000 = 2^32
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-19-2007 06:12 AM
11-19-2007 06:12 AM
Re: Simple multiplication gives incorrect results
(When you multiplied by 512 you were calclating the number of free bytes, not megabytes.)
FS = f$getdvi("DISK1","FREEBLOCKS")
MB = FS/2048
1 Block = 512 Bytes = 0.5 KiloBytes
1 MegaByte = 1024 KiloBytes
And welcome to the IT resource center forums.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-19-2007 06:27 AM
11-19-2007 06:27 AM
Re: Simple multiplication gives incorrect results
That's normally called 32-bit signed
arithmetic. One of the 32 bits is a sign
bit.
I don't recall a comp.os.vms question this
year from anyone wondering why some simple
floating-point calculation wasn't exact. Is
this as close as we get for 2007?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-19-2007 06:33 AM
11-19-2007 06:33 AM