Discussion about math, puzzles, games and fun.   Useful symbols: ÷ × ½ √ ∞ ≠ ≤ ≥ ≈ ⇒ ± ∈ Δ θ ∴ ∑ ∫ • π ƒ -¹ ² ³ °

You are not logged in.

• Index
•  » Help Me !
•  » Computer Nerd needing Math Nerd Help - SciNotation?

|
Options

campbeln
2005-08-08 16:52:32

On a bit of an aside... is there any significance to these numbers in relation to base 16? These datatypes range limitations are bread from having to fit X number into Y bytes (hence ranges like 0 to 255 and -32768 to 32767), yet these numbers don't seem to conform to this (or do they)? Of course there is some overhead to store (decimal position, etc) but still...

###########
You can disregard the rest of  this, these datatypes have the ability to have variable precisions, hence the discrepancies!
###########

Also... there is a term that I've run into in relation to SQL specifications referred to as "numeric precision" that is causing some friction with the above answers. As you may have noticed above, I have a JavaScript function that calculates this value which is basically the number of places in the min/max value. Or, as it's defined online...

* Total number of digits for exact numeric data types (DECIMAL, INTEGER, MONEY, SMALLINT)
* Number of digits of mantissa precision (machine-dependent) for approximate data types (FLOAT, SMALLFLOAT)

Now, the best part of this is that...

* The first range [-(10^38 + 1) to (10^38 - 1)] has a "numeric precision" of 18 (or 20/21 places to few)
* The second range [(-3.40e + 38) to (3.40e + 38)] has a "numeric precision" of 24 (or 14/15 places to few)
* The third range [(-1.79e + 308) to (1.79e + 308)] has a "numeric precision" of 53 (a lot of places to few)

Does anyone know what is being referred to by this "numeric precision" (it is part of SQL92’s “Information Schema” definition)? How should I mitigate the seeming rift between these two different definitions of the datatypes ranges?

MathsIsFun
2005-08-08 16:05:07

#### campbeln wrote:

But 4 of my questions still remain... can anyone provide the integers for:

(-3.40e + 38) = ?
(3.40e + 38) = ?

(-1.79e + 308) = ?
(1.79e + 308) = ?

Just a matter of making sure we have enough zeros!

-340000000000000000000000000000000000000
340000000000000000000000000000000000000

-1.79e + 8 = 179000000, so -1.79e + 308=
-179000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Likewise 1.79e + 308 =
179000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

I counted the characters in the last one and got 309, as expected, because 1 x 10¹ has two characters (10).

campbeln
2005-08-08 15:59:01

wcy - nice... but how will that help me? Converting into Hex (base 16) would get me a "shorter" number, in terms of a string but I'd still run into the same issues when it comes to variable widths. Though again, math and I have never really gotten along, so I could be missing something!

Thanks for all the responses, all! But 4 of my questions still remain... can anyone provide the integers for:

(-3.40e + 38) = ?
(3.40e + 38) = ?

(-1.79e + 308) = ?
(1.79e + 308) = ?

Muchos Danke!

ganesh
2005-08-08 15:14:41

wcy, i viewed the base converter. It works well.
I tried converting 759 to base 20 and got the right answer

wcy
2005-08-08 15:07:02

i created a base converter before,
maybe it might be useful to convert your integers to other bases (eg base 16)

http://www.geocities.com/chengyuanwu/baseconverter.html

campbeln
2005-08-08 11:55:28

Jason - please post a new topic by going here - http://www.mathsisfun.com/forum/post.php?fid=2

jason
2005-08-08 11:47:13

is anyone here

jason
2005-08-08 11:46:00

i need help with a riddle my teacher gave me
its m168a:00th
3 words
1 word is math
: means time

campbeln
2005-08-08 11:44:21

kylekatarn - thanks for that!
I have a feeling the "notation" used in the documentation for SQL*Server is not 100% correct, mathematically or otherwise. I have a feeling they did indeed mean "-(10^38 + 1)" as negative numbers are allowed in the datatype. Microsoft just failed to document it correctly (us computer nerd's are used to having their documentation be incorrect; completely non-functional example code is my personal fav.!).

campbeln
2005-08-08 11:35:00

For those who are interested, here is an updated version of the JavaScript code I am going to use to test these huge numbers. The previously posted function assumed that the passed sRangeMin was <= 0. This version should no longer have that limitation ("should" being the operative word!).

Again, anyone who is willing to look thru my code to ensure my logic is correct I would be greatly appreciative!

//############################################################
//# Determines if the passed sNumber is within the passed range
//#    NOTE: "return (sNumber >= sRangeMin && sNumber <= sRangeMax)" would work in 99.9% of the checks we'll do with this function, but in the case of huge/tiny numbers (such as NUMERIC(x,y)'s in Oracle), this wouldn't cut it as the numbers would be too large/small to be represented in any available numeric variables
//############################################################
//# Last Updated: August 8, 2005
function rfIsNumberInRange(sNumber, sRangeMin, sRangeMax) {
var bReturn = false;

//#### If the passed sNumber is greater then or equal to the passed sRangeMin
if (rfCompairson(sNumber, sRangeMin) >= 0) {
//#### If the passed sNumber is less then or equal to the passed sRangeMax
if (rfCompairson(sNumber, sRangeMax) <= 0) {
//#### Since the passed sNumber is within the passed sRangeMin/sRangeMax, flip our bReturn value to true
bReturn = true;
}
}

//#### Return the above determined bReturn value to the caller
return bReturn;
}

//############################################################
//# Determines if the passed sNumber is greater then, less then or equal to the passed sRange
//#     NOTE: Return values:
//#          -1 if sNumber is less then sRange
//#          1 if sNumber is greater then sRange
//#          0 if the passed values are equal, or if one of the passed values was non-numeric
//############################################################
//# Last Updated: August 8, 2005
function rfCompairson(sNumber, sRange) {
//#### Ensure the passed sNumber and sRange are strings
sNumber = new String(sNumber);
sRange = new String(sRange);

//#### Define and init the required local vars
var iNumberNumericPrecision = rfNumericPrecision(sNumber);
var iRangeNumericPrecision = rfNumericPrecision(sRange);
var bNumberIsPositive = (sNumber.indexOf("-") != 0);
var bRangeIsPositive = (sRange.indexOf("-") != 0);
var iReturn;

//#### If the passed sNumber or sRange were non-numeric, set our iReturn value to 0
if (iNumberNumericPrecision == -1 || iRangeNumericPrecision == -1) {
iReturn = 0;
}
//#### Else if the signs of the passed sNumber and sRange do not match
else if (bNumberIsPositive != bRangeIsPositive) {
//#### If the bNumberIsPositive, then the sRange is negetive, so set our iReturn value to 1 (as sNumber is greater then the sRange)
if (bNumberIsPositive) {
iReturn = 1;
}
//#### Else the sNumber is negetive and the bRangeIsPositive, so set our iReturn value to -1 (as sNumber is less then the sRange)
else {
iReturn = -1;
}
}
//#### Else the signs of the passed sNumber and sRange match
else {
//#### If the above-determined rfNumericPrecision's are specifying numbers of less then 1 billion
if (iRangeNumericPrecision < 10 && iNumberNumericPrecision < 10) {
//#### Define and init the additionally required vars
//####     NOTE: We know that both sNumber and sRange are numeric as non-numeric value are caught by rfNumericPrecision above
var fNumber = Math(parseFloat(sNumber));
var fRange = Math(parseFloat(sRange));

//#### If the sNumber and sRange are equal, set our iReturn value to 0
if (fNumber == fRange) {
iReturn = 0;
}
//#### Else if the sNumber is greater then the sRange, set our iReturn value to 1
else if (fNumber > fRange) {
iReturn = 1;
}
//#### Else the fNumber is less then the sRange, so set our iReturn value to -1
else {
iReturn = -1;
}
}
//#### Else we're dealing with number ranges over 1 billion, so let's get creative...
else {
//#### If the iNumber('s)NumericPrecision is less then the iRange('s)NumericPrecision (making sNumber less then sRange), set our iReturn value to -1
if (iNumberNumericPrecision < iRangeNumericPrecision) {
iReturn = -1;
}
//#### Else if the iNumber('s)NumericPrecision is more then the iRange('s)NumericPrecision (making sNumber greater then sRange), set our iReturn value to 1
else if (iNumberNumericPrecision > iRangeNumericPrecision) {
iReturn = 1;
}
//#### Else the iNumber('s)NumericPrecision is equal to the iRange('s)NumericPrecision, so additional checking is required
else {
//#### Define and set the additionally required decimal point position variables
var iNumberDecimalPoint = sNumber.indexOf(".");
var iRangeDecimalPoint = sRange.indexOf(".");

//#### If either/both of the decimal points were not found above, reset iNumberDecimalPoint/iRangeDecimalPoint to their respective .lengths (which logicially places the iRangeDecimalPoint at the end of the sCurrentRange, which is where it is located)
//####    NOTE: Since this function only checks that the passed sNumber is within the passed range, the values "whole" -vs- "floating point" number distinction is ignored as for our purposes, it is unimportant.
if (iNumberDecimalPoint == -1) {
iNumberDecimalPoint = sNumber.length;
}
if (iRangeDecimalPoint == -1) {
iRangeDecimalPoint = sCurrentRange.length;
}

//#### If the sNumber's decimal point is to the left of sRange's (making sNumber less then sRange), set our iReturn value to -1
if (iNumberDecimalPoint < iRangeDecimalPoint) {
iReturn = -1;
}
//#### Else if the sNumber's decimal point is to the right of sRange's (making sNumber greater then sRange), set our iReturn value to 1
else if (iNumberDecimalPoint > iRangeDecimalPoint) {
iReturn = 1;
}
//#### Else the sNumber's decimal point is in the same position as the sRange's decimal point
else {
//#### Define and init the additionally required vars
var iCurrentNumberNumber;
var iCurrentRangeNumber;
var i;

//#### Default our iReturn value to 0 (as only > and < are checked in the loop below, so if the loop finishes without changing the iReturn value then the sNumber and sRange are equal)
iReturn = 0;

//#### Setup the value for i based on if the bNumberIsPositive (setting it to 0 if it is, or 1 if it isn't)
//####    NOTE: This is done to skip over the leading "-" sign in negetive numbers (yea it's ugly, but it works!)
//####    NOTE: Since at this point we know that signs of sNumber and sRange match, we only need to check bNumberIsPositive's value
i = (bNumberIsPositive) ? (0) : (1);

//#### Traverse the sNumber/sRange strings from front to back (based on the above determined starting position)
//####     NOTE: Since everything is is the same position and the same precision, we know that sNumber's .lenght is equal to sRange's
for (i = i; i < sNumber.Length; i++) {
//#### As long as we're not looking at the decimal point
if (i != iNumberDecimalPoint) {
//#### Determine the iCurrentNumberNumber and iCurrentRangeNumber for this loop
iCurrentNumberNumber = parseInt(SubStr(sNumber, i, 1));
iCurrentRangeNumber = parseInt(SubStr(sRange, i, 1));

//#### If the iCurrentNumberNumber is less then the iCurrentRangeNumber
if (iCurrentNumberNumber < iCurrentRangeNumber) {
//#### sNumber is less then sRange, so set our iReturn value to -1 and fall from the loop
iReturn = -1;
exit;
}
//#### Else if the iCurrentNumberNumber is greater then the iCurrentRangeNumber
if (iCurrentNumberNumber > iCurrentRangeNumber) {
//#### sNumber is greater then sRange, so set our iReturn value to 1 and fall from the loop
iReturn = 1;
exit;
}
}
}
}
}
}
}

//#### Return the above determined iReturn value to the caller
return iReturn;
}

//############################################################
//# Determines the numeric precision of the passed sValue (i.e. - counts how many numeric places there are within the number, not including leading 0's)
//############################################################
//# Last Updated: August 8, 2005
function rfNumericPrecision(sValue) {
var sCurrentChar;
var iReturn = 0;
var i;
var bStartCounting = false;

//#### Ensure the passed sValue is a string
sValue = new String(sValue);

//#### If the passed sValue is not a number, set the iReturn value to -1 (which indicates an error occured)
if (isNaN(sValue)) {
iReturn = -1;
}
//#### Else the passed sValue is a number
else {
//#### Traverse the .length of the passed sValue
for (i = 0; i < sValue.length; i++) {
//#### Collect the sCurrentChar
sCurrentChar = sValue.substr(i, 1)

//#### If the sCurrentChar is a number
if (! isNaN(sCurrentChar) && sCurrentChar != ' ') {
//#### If we are supposed to bStartCounting, or if the sCurrentChar is not a 0
//####    NOTE: This is done so we ignore leading 0's (trailing 0's are still counted)
if (bStartCounting || sCurrentChar != '0') {
//#### Inc iReturn and ensure bStartCounting is true
iReturn++;
bStartCounting = true;
}
}
}
}

return iReturn;
}

kylekatarn
2005-08-08 11:32:18

Nothing is wrong with the calculator!

(-10)^38 is indeed 100000000000000000000000000000000000000.

x^n =x*x*x*x* ... *a    (n times)
If n is EVEN then x^n is greater or equal to zero:
If n is ODD then x^n can be a negative number.

a simple example:
(-10)^6=(-10)(-10)(-10)(-10)(-10)(-10)=1000000
(-5)^3=(-5)(-5)(-5)=-125

you were probably thinking of -(10^38)

campbeln
2005-08-08 10:16:31

Thanks Admin! Seems I was missing a digit in the (-10^38 + 1) to (10^38 - 1). And just so you know, in the Full Precision Calculator:

X = -10
Y = 38
[X^(Integer Y)]
Returns (100000000000000000000000000000000000000)

...shouldn't this return (-100000000000000000000000000000000000000)?

'Course I've still got the other two ranges to determine.

Also... I was just reviewing my JavaScript function and I've noticed a few bugs which I'm now working on (so be gentle =).

MathsIsFun
2005-08-08 10:05:39

I can only spare a minute, and will read your post later, but just thought I would mention Full Precision Calculator

For example: put 10 into X and 38 into Y, then calculate X^Y, then copy-paste this into X, put 1 in Y and use X-Y

campbeln
2005-08-08 09:41:13

Hello all,
I am interested in getting the integer values (well, the long number versions) for the following values in scientific notation:

(-10^38 + 1) = -10000000000000000000000000000000000001 [correct?]
(10^38 - 1) = 9999999999999999999999999999999999999 [correct?]

(-3.40e + 38) = ?
(3.40e + 38) = ?

(-1.79e + 308) = ?
(1.79e + 308) = ?

I apologize for the mundane topic, but my guile has run dry in trying to find the numeric equivalents to these values (stupid Excel and its rounding). Thanks for any help you can provide! My approach to math always seemed to be about 180 degrees from the way it was taught to me, so I rarely excelled at it. In fact, my altered approach is made evident in the method I take to validate these huge numbers (more info for those interested parties below...)!

#########
For those who are interested in why I'm asking this question...
#########
I am working on a website front end script that validates user input to ensure that it will "fit" into a back end database. The values listed above are the valid numeric ranges allowed for three data types (FLOAT, REAL and DECIMAL column datatypes in SQL*Server in this case).

Now those of you with some computer nerd experience may be asking "How the heck is he planning on validating these huge numbers!? Sci-Notation is used for a reason!" Well, I have built a function (both in JavaScript and VB.Net) that walks these huge numbers a single digit at a time, hence avoiding the "hugeness of the numbers" issue (I've included the JavaScript version below for those of you who are truly interested - actually if you could make sure I've done it right that would rock!). In a nutshell - by counting the number of digits, then the position of the decimal places and finally by looking at each digit's value individually I am able to use simple, small number comparisons to determine if these truly huge numbers are in the proper range.

#########

[Clipped original, limited JavaScript code. Please see below for latest code.]