What about a binary multiplication table? Which leads to division. And then decimal places. 

4 hours ago, Brony Number 42 said:

What about a binary multiplication table? Which leads to division. And then decimal places. 

Well, you can have fractional numbers in binary as well, just that they are not exactly used like that.

Normally, yea, you could represent one half as 0.1, one quarter as 0.01 and three quarters as 0.11.

However, in computers, fractional numbers come in two types - fixed point and floating point. Fixed point, as the name suggests, has the decimal point in the same place all the time. It's essentially an integer that you always divide by some number (for example, you count cents, then divide that by 100 and display it as Euros). This is useful for operations involving money and similar where accuracy is important (more on that later), but you do not need to move the decimal point arbitrarily.

Floating point numbers come in a few types, but they all work like the "scientific notation" works in decimal. Just like you can represent 0.034 as 3.4*10^-2 or 3.4e-2, the same is done in binary - there is a part of the variable that shows the "value", called mantissa and a part which shows where to put the decimal point, called the exponent. The space for those parts is limited, so you get limited precision - adding 1e10 + 1e-20 you will lose the small part, since the mantissa may not have enough space to represent the whole value. Just imagine having a limited amount of space to write the numbers after the decimal when writing the format 1.2345e5. 

So, when you do operations with floating point numbers, you lose a little bit of precision, which makes it a problem if you use floating point to calculate money. Floating point has another problem - just like in decimal, some numbers have infinite number of digits after the decimal. Representing 0.1 (decimal) in binary is like representing 1/3 in decimal. So, with floating point, 0.1*2 may be a little different than 0.2.

This can be shown in decimal as well. 1/3 is 0.(3), but if you had limited amount of space and no way to signify that the 3 is repeating, you would have tow write it as 0.3333. Now, multiply that by 3 and you get 0.9999, but the real value is 1. Even better, 2* (1/3) = 2* 0.3333 = 0.6666, while 2/3 = 0.6667.

Let's do some binary division! 

@Pentium100 Yes, I know everything that I need to know about fixed point. Sadly, floating point has failed me too many time to make me like it, like you said with the precision. But floating point also has issues representing whole numbers, as well as testing for whole numbers (equal to exactly a whole number).

I don't know the scientific notation (because it has never been useful to me), so that's probably why I fail to completely understand floating point.

And then there is the even worse situation of floating point, and it is running the same math on different systems. I have had many computers with ATI and Nvidia running GPU code for me, where texture coordinates are very important, and those texture coordinates are normalized (between 0.0 and 1.0, no matter the texture size). And the numbers are stored as a format called "half", a 16 bit floating point value that is controlled internally by the graphics card. It turned out that I got texture errors on Intel graphics cards, which, to me tells a story how Intel is bad at math where ATI and Nvidia are not? That doesn't make sense!
I also had 1 computer running a Pentium 4 that had completely different floating point results, and completely ruining my game. Again, making Intel look horrible at handling math. Shouldn't Intel know math is important?

But fixed point has never been a problem for me..... Except using square root, where I lose so much precision compared to floating point, and there is nothing I can do!

@Brony Number 42 I will get to division one of these days. Sadly, it is the thing that works really oddly with binary. Especially when working with negative numbers. I will go into negative numbers when we talk about subtraction.

@Splashee Division is not odd, it works exatly the same. Unless computers do something odd. I don't know enough about computers.

Just now, Brony Number 42 said:

@Splashee Division is not odd, it works exatly the same. Unless computers do something odd. I don't know enough about computers.

Well, you know about long division, and remainder, right?
Because when we are dealing with binary, we are doing whole numbers, no decimal point stuff. That means we will have to round the answer. There is also the issue of Divide by Zero, and negative numbers, and how slow division is (it is one if not the slowest operation a computer can make).

So you are talking about computers, not pure math? Because there are decimals in binary (or any base) division. But maybe I'm getting ahead of things.

What is pure math?

Computers can do divisions on decimal point number, you know floating point numbers @Pentium100 was talking about. Floating point can also have -0.0 which is a number binary cannot have, only +0.
 

I am not the best at math, to be honest. I am expecting to fail some of my calculations at one point. I am trying to double check everything i do, and triple check as well.

Pure math means theoretical math. Pencil and paper. Humans doing math. Not how computers do math. 

You can have binary division, something like 1001/110 and get an answer with decimals. I know how to do that on paper, but I don't know exactly how a computer does it.

How do you do it on paper?
I actually can't do 9/3. I can in my head because I know it is 3 (multiplication table), but I don't know of a way to divide with base-2.

6 hours ago, Splashee said:

I don't know the scientific notation (because it has never been useful to me), so that's probably why I fail to completely understand floating point.

It's quite simple really. Instead of writing lots of zeros (0.0000003221 or 123400000000) where it's easy to miscount them, you can write it as such:
Move the decimal point until only one non-zero digit is in front of it (so, you get 3.221 and 1.234) and count how many spaces you moved it (7 and 11). If you moved the decimal point to the right, then make the count negative (-7 and 11) and write out the number, so you get 3.221e-7 and 1.234e11 or 3.221*10^-7 and 1.234*10^11.

Essentially what that means is that you take the "value", 3.221 and multiply it by 10^-7, which is 0.0000001 to get the number you want.

3.221e-7 = 32.21e-8 = 322.1e-9

Engineering notation is essentially the same, except the mantissa can be between 1 and 1000 (scientific notation it's between 1 and 10) with the exponent restricted to multiples of three (to match with the kilo-, mili- etc prefixes), so I can write the capacity of 470uF as 470e-6F or 4.7e-4F.

It's super easy. Barely an inconvenience.

Consider an example 430/13. Writing this as long division, as best I can with the type set here. Writing some 0s to the right, since we will need them. 43 is the dividend, 13 the divisor, and the answer is the quotient.

13 | 430.000

I start at the first digit of the divisor and move right until I get a number bigger than the dividend. 4 is not bigger, but 43 is. This means 13 can go into 43 at least once, but not mpre than, 9 times. At this point, you can find a number  between 1 and 9 to multiply by 13 to get as close to 43 without goimg over, like Price Is Right. It is 3, so I write that on top as the first digit of my answer. Multiply 3 ×13 and subtract that from 43

          3
13 | 430.000
    -   39
           4
Bring down a 0 and repeat the process. The next number is 3 again
 
         3              
13 | 430.000
    -     39
           40.

          33.         
13 | 430.000
    -     39
           40.
-           39
               10
13 cannot go into 10, so I put 0 for the next number and bring down a 0

          33.0        
13 | 430.000
    -     39
           40.
-           39
               100
7×13= 91 so 7 is my next number

          33.07        
13 | 430.000
    -     39
           40.
-           39
               100
    -               91
                     9
This can keep going.
 

Floating point is very similar to the scientific notation, except in binary.

So, you want to represent 0.00134277343, you can write it in binary as 0.0000000001011, but, like with decimal, this takes up a lot of space for very large and very small numbers because of the leading or trailing zeros. 

Using the same rules for scientific notation (but in binary) you can represent that number as 1.011 * 2^-10, which takes up less space. This is how it is done in floating point - you have one bit for the sign, some bits for the exponent (the -10) and some bits for the fraction. Since there is always a 1 in front of the decimal point (because you move the point until there is 1 in front of it), it is not stored, so this could be stored as a floating point variable as

sign = 0b0 (positive)
exponent = this is stored as 127+e, so 127-10 = 117 = 0b01110101
fraction = 0b0110000
so, you get a 16bit float as 0b0011101010110000 = 0x3AB0

There are special values for zero, infinities and NaN. By the way, ones' complement integers also have negative zero.

The problems with precision come down to the limited space for the fraction, just like it would take a lot of space to write 1420000000000.000123, it does take a lot of space to write that in binary. It's all about significant digits, just like if somebody 5 years ago told you that some fossil is 100 million years old, it does not mean that the fossil now is 100 million and 5 years old, since the original age of the fossil probably was not determined with the precision of one year.

Or course the problem with floating point and how we use it is also because "normal" decimal numbers are not possible to fully represent in binary. For example, 0.3 in binary would be 0.(0011), so, no matter how big a floating point variable you use, you will not be able to represent 0.2 completely accurately, like you can represent 0.125 or similar. Just like representing 1/3 in decimal = 0.(3), sooner or later you will run out of space for the digits and the number still won't be completely precise.

@Brony Number 42 Yea, long division is..... very long! Nice explanation, I wish I had gotten that in school!

From what I have seen, the computer way of doing division looks really odd. In order for me to figure it out, I have to compile a C program for a computer that doesn't have divide, and then find the code which is forced to be there to do the division. After all these years, I have never needed any tricks for doing divisions that aren't a power of 2.

I can show you an example of my problem I am facing right now. Imagine you have a large grid, well larger than the positions you can be inside each grid. There are 16 steps in each grid, until you get to the next grid number. Let's draw the grid here:

[-4] [-3] [-2] [-1] [0] [1] [2] [3] [4]

If you are at position 0, you are in grid [0]. If you are at position 15, you are still in grid [0]. As soon as you are at position 16, you are in grid [1].
Obviously, by looking at the order of the grid, if you are at position -1, you should be in grid [-1]. Computer division by binary shifting will give you that answer! But pure math doesn't:

Position 0 divided by a grid size of 16, gives 0 (truncated, only whole numbers, not caring about remainder for now!)
Position 15 divided by a grid size of 16, gives 0 (still holding true to our rules above)
Position 16 divided by a grid size of 16, gives 1 (still holding true to our rules above)
Position -1 divided by a grid size of 16, gives 0 (NO, here is where it breaks. You can see it should have given -1, but it gave 0 instead)
Position -15 divided by a grid size of 16, gives 0 (Still wrong. The grids are clearly numbered, there is no duplicate [0] grid)

Computer bit shifting is really quick, it gives the right answer, such as -15 shifted to the right by 4 (divided by 16) gives -1.
 

I understand bit shifting but I'm not sure what you mean by this grid.

The binary multiplication table is 

      0   1   10

0    0   0     0

1    0   1   10

10  0   10  100

So for example 

1001 / 101 = 1.1100 ... with more digits

4 hours ago, Brony Number 42 said:

13 | 430.000

Interesting, the way I learned this in school was the opposite, that is, if I want to divide 430/13, I would have to write 

430 | 13

(yes, the 13 is in a "corner" and IIRC we called this "dalyba kampu" which literally means "division by corner"). The result is written under the divisor.d):

29 minutes ago, Splashee said:

If you are at position 0, you are in grid [0]. If you are at position 15, you are still in grid [0]. As soon as you are at position 16, you are in grid [1].
Obviously, by looking at the order of the grid, if you are at position -1, you should be in grid [-1]. Computer division by binary shifting will give you that answer! But pure math doesn't:

This is because your positions are shifted . If you put the "0" position in the middle of the grid (or rather, have -1 and +1 positions near the center of the grid or divide the grid into odd number of positions so that 0 can be in the center), then you could get the grid value as position/9. (0 -> 0, +8 -> 0, -8 -> 0, +9 ->1, -9 -> -1)

Your way, however, works great in computers with twos' complement integers as long as the bit shift operation keeps the MSB in place.

@Pentium100 Wait, how do I read binary after the decimal point?

I assumed something like for the first position, it would be 1/2 (0.5), then 1/4 (0.25), then 1/8 (0.125)... But adding those together will never get 0.2 or 0.3.

I can guess forever, like, maybe reverse, 1 divided by the sum of the positions in reverse?

29 minutes ago, Splashee said:

Wait, how do I read binary after the decimal point?

I assumed something like for the first position, it would be 1/2 (0.5), then 1/4 (0.25), then 1/8 (0.125)... But adding those together will never get 0.2 or 0.3.

Exactly like that, yes. 0.2 is a recurring fraction in binary, just like 1/3 is in decimal.

so it is 0.001100110011... forever.

@Splashee Don't try convert between decimal and binary unless you need to. If you see 0.00110011 in binary, then just think of it as that, don't worry about what it might be in a different base. If you need to convert then do this

From binary to decimal remember 0.1 = 1/2 , 0.01 = 1/2^2 , 0.001 = 1/2^3 etc, and add them up so 0.101 bin = 1/2 + 0/2^2 + 1/2^3 dec

From decimal to binary remember 0.1 = 1/10 , 0.01 = 1/10^2 , 0.001 = 1/10^3  and 10 dec = 1010 bin

so 0.23 dec, convert each to bin

0.23 dec =2 / 10 + 3 / 10^2 dec =  10 / 1010 + 11 / 1010^10 bin. Then you have to do the binary long division on each piece and add them = 0.001110101.... bin. It is not difficult, just tedious with all the multiplication and long division in binary. For example 1010^10 bin = 1100100 bin 

3 hours ago, Splashee said:

I assumed something like for the first position, it would be 1/2 (0.5), then 1/4 (0.25), then 1/8 (0.125)... But adding those together will never get 0.2 or 0.3.

It is exactly like that. And 0.2 or 0.3 in binary is like 1/3 or 1/11 in decimal - repeating forever. This is why floating point cannot completely precisely represent these values.

14 hours ago, Brony Number 42 said:

I understand bit shifting but I'm not sure what you mean by this grid.

The binary multiplication table is 

      0   1   10

0    0   0     0

1    0   1   10

10  0   10  100

So for example 

1001 / 101 = 1.1100 ... with more digits

By knowing the multiplication table, you can just follow the rules of long division to get the answer? Is it that simple?

It will be very fun when it is time for me to explain division of binary numbers. My goal is to write as clear and easy way to do the operations as possible.

40 minutes ago, Pentium100 said:

It is exactly like that. And 0.2 or 0.3 in binary is like 1/3 or 1/11 in decimal - repeating forever. This is why floating point cannot completely precisely represent these values.

Quote

0.3 in binary would be 0.(0011)

Just to be clear, 0.0011 would be (1/8 + 1/16)? Wouldn't that be 0.1875 in decimal? Am I off by one digit? Or am I reading your numbers or parenthesis wrong?

I noticed that I get closer to 0.3 decimal if I do (1/4 + 1/16), which is binary 0.0101? Decimal 0.3125.

1 minute ago, Splashee said:

Just to be clear, 0.0011 would be (1/8 + 1/16)? Wouldn't that be 0.1875 in decimal? Am I off by one digit? Or am I reading your numbers or parenthesis wrong?

 

I noticed that I get closer to 0.3 decimal if I do (1/4 + 1/16), which is binary 0.0101? Decimal 0.3125.

When doing this, you can never go over the original number and 0.3125 is closer, but it's too big.

Oh, and I made a mistake, I converted 0.2 to binary, which is 0.(0011), but wrote 0.3 Oops...
0.3 in binary would be 0.01(0011) 

0.0011 would be 0.1875. The parenthesis indicate that the part is repeating forever, so 0.2 = 0.001100110011001100110011...

0.0011 = 0.1875
0.00110011 = 0.19921875
0.001100110011 = 0.199951172

And for 0.3 (for real this time) it's:
0.01 = 0.25
0.010011 = 0.296875
0.0100110011 = 0.299804688
0.01001100110011 = 0.299987793
and so on. If you kept adding the ever smaller fractions to infinity, you would end up with 0.3

Just like trying to write 1/11 as a decimal fraction you end up with 0.(09).

Now, you can round up those numbers, for example, say that 1/11 = 0.09091 or that 0.2 in binary is 0.0011001101, but it's still not precise. So, just liek it's impossible to write 1/11 as a decimal fraction precisely in a finite amount of space, so is writing 0.2 as a binary fraction. 

indeed. the 1/11 and 1/101 are both recurring fractions in binary (although of course 1/3 is and 1/5 isn't in decimal)

Just for clarification, "1/11" in my previous post was used as an example of decimal recurring fraction, one divided by eleven.

@Pentium100 Your explanation was flawless this time! I really understand the repeating forever part. Here is where 64 bit (double) floating point get closer to the real number, than a 32 bit float, because it has more of the fraction bits, getting closer to the impossible to reach number.

I have been doing quite a lot of work with floating point numbers, and I have gotten serious headaches in the process. I think most of them are hardware specific problems. And i don't have time to invest time into individual cases. But knowing this will surely help! Thanks! And thank you @Brony Number 42 for your excellent math knowledge as well. And hi @CypherHoof!