Math Is Fun Forum
  Discussion about math, puzzles, games and fun.   Useful symbols: ÷ × ½ √ ∞ ≠ ≤ ≥ ≈ ⇒ ± ∈ Δ θ ∴ ∑ ∫ • π ƒ -¹ ² ³ °

You are not logged in.

#26 2010-12-16 12:36:49

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Hi bobbym
Instrument 1 is responding more to the 50% and a lower 90% but then at around Time 5, there is an increase in the 90% followed by a decline in Instrument 1 and less of a decline in Instrument 2. To me that would suggest Instrument 2 is more sensitive to the 90%. As for post-time step 15, see comment #27.

A tricky problem to try and index I guess! Apologies for taking up a lot of your time today.

Offline

#27 2010-12-16 12:40:57

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

No problem.

It is not over yet. Actually we are just starting.

One question, why do you want to use the geometric mean?


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#28 2010-12-16 12:49:43

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Mainly as to my understanding it's quite useful for summarizing highly skewed data, which I suspect this is.

Offline

#29 2010-12-16 13:02:45

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Would it be acceptable if I try to fit a relationship between the reading on instrument one and 2 to the 90%.

And then fit another to the 50% and you could analyze later what the 2 meant together?

Or are the 50% and 90% related? If so how?


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#30 2010-12-16 13:09:47

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

By all means bobbym! Unfortunately though the 50 and 90% are related in that they describe the 50th and 90th percentile of the particle size range for each sample (I also have the 10th percentile but that does not have much effect, physically speaking). I have done some plots looking at the relationship between individual instruments and the 50 and 90% and they are pretty scattered - the issue here is that the millivolt output is not just responding to the 50 and 90th but everything! However, the 90th will have more of an influence given the optics and wavelength of each instrument. That probably doesn't help much!

Offline

#31 2010-12-16 13:19:42

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Hi toitu;

There may not be enough of an effect between the instruments and the 90%. I will not know until I start to try to fit something. Do you have any data in terms of average sizes
rather than a percentile?


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#32 2010-12-16 13:30:09

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Unfortuntaley I only have the percentile values so I guess the 50th percentile represent the median particle size in microns rather than the mean.

Offline

#33 2010-12-16 13:35:17

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Hi;

That is unfortunate, the median contains less of the character of a sequence or sample than the mean does. That may be the reason for the wild fluctuations. The mean might have smoothed things out by a lot.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#34 2010-12-16 14:56:33

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Fitting half the problem now. Responding better than I thought it would. Now the question is how close is acceptable to you?


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#35 2010-12-16 15:38:22

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Hi Bobbym
If the fit is reasonable, I'll be pretty happy - I'm not looking for an exact fit as not possible with this type of data but if we're in the vicinity of 50 to 60% of the variance explained, that works for me!

Offline

#36 2010-12-16 15:41:46

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Right now I am struggling with the residuals they are a little too large. That particular value might be an anomaly, or error in measurement. I will not know until I try more fits.

It is going to be a while. Going to take a little break to eat.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#37 2010-12-16 15:46:00

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

There are indeed a couple of large departures in the dataset that are a little difficult to decide if they are 'real' or not. Thanks for all your efforts on this, it is much appreciated.

Offline

#38 2010-12-16 20:02:28

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Hi toitu;

Sorry, I could not get back to you sooner, but my internet went out for 2 1/2 hours. The largest error I have now is around 35. My predicted value is off by 35  from one of your data values. The rest are close I am still trying to reduce these residuals as they are called down. Please hold  on it will take time.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#39 2010-12-17 03:28:02

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Hi bobbym
that sounds pretty good!

Offline

#40 2010-12-17 04:11:33

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

I was able to fit a relationship between all 4 parameters. It is large but not too difficult to work with.

i1 is the first meter reading.
i2 is the second meter reading.
f is the fifty percentile data.
n is the ninety percentile data.

The equation is:

Where:

This has a maximum error of 15.45 for any particular point. Most are a lot less. It is fairly obvious from the residuals that a better fit is still possible.

Please allow me some time to check out the values. Lots of copying can result in errors.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#41 2010-12-17 04:27:07

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

That looks fantastic bobby, thanks so much. When you are happy with the fit/residuals, would
you be able to explain to me what you did to get to this point? Thanks

Offline

#42 2010-12-17 04:48:08

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Hi;

Please give me lots of time to check the figures. There are undoubtedly typos.

would
you be able to explain to me what you did to get to this point?

It is totally an industrial strength solution. Explaining it is harder than doing it. Lots of computing and lots of guesswork. There is one piece of math in here that is unique.

Long time ago in galaxy far, far away, I happened to discover a method of curve fitting that achieved the much desired minimax fit. It is fairly common for continuous data but I never saw anything like my method for discrete points. I went around for a long time using it and hiding my discovery. I named it ABput and used it to compress an entire poker simulation into several formulas. You can imagine my sorrow when I finally found out that a great numerical analyst named Remez had not only done it first but did it better.

You will be happy to know I used it on your problem.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#43 2010-12-17 04:50:59

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Sounds quite interesting (and complex!). I guess I have some reading ahead of me!

Offline

#44 2010-12-17 05:01:41

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Hi toitu;

Will be taking a little break now to rest my eyes. I will resume work on it later. Then we can talk about trying to use effectively, alse seeing whether it has any predictive power...


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#45 2010-12-17 05:08:13

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Hi bobbym, sounds great! Thanks again

Offline

#46 2010-12-18 00:18:31

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Hi toitu;

Have run into some problems, a certain piece of software is doing the calculation of the residuals in an unusual way. It is probably not wrong but just not what I expected it to do. This has slowed the work down. A whole day and a half has been wasted. Now I must start again. I am sorry.

Some new relationships have been found.

Where :

f is the fifty percentile data
i1 is instrument or meter 1
i2 is instrument or meter 2

This relation has a maximum error or 4.54.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#47 2010-12-18 06:32:41

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Hi bobbym, no need to apologize at all! That's a pain the software is not responding properly. However, residuals of 4.54 um is impressive!

Offline

#48 2010-12-18 10:47:22

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

Okay, to the next problem. You now have two relationships, one between all the variables and one between the 2 meters and the 50 pecentile data.

To get a relationship between just the 2 meters and the 90 percentile data seems obvious now but the obvious things are always wrong.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

#49 2010-12-18 13:14:57

toitu
Member
Registered: 2010-03-16
Posts: 32

Re: Creating a dimensionless index

Hi bobbym, I agree that using only the 2 instruments to predict the 90 will be difficult. Especially as under normal circumstances we wouldn't be able to 'temper' the instrument with the 50 percentile data as usually we wouldn't know this. I will look foward to hearing how you get on with this last step. Thank you so much for your help, you have gone way beyond expectations on the amount of time you've spent on this.

Offline

#50 2010-12-18 14:51:02

bobbym
bumpkin
From: Bumpkinland
Registered: 2009-04-12
Posts: 109,606

Re: Creating a dimensionless index

I have not tried the obvious move yet but it might be possible to smooth the data or even eliminate one bad data point.


In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Offline

Board footer

Powered by FluxBB