Discussion about math, puzzles, games and fun.   Useful symbols: ÷ × ½ √ ∞ ≠ ≤ ≥ ≈ ⇒ ± ∈ Δ θ ∴ ∑ ∫  π  -¹ ² ³ °

You are not logged in.

## #1 Re: Help Me ! » Periodicities and Fourier analysis? » 2012-06-01 11:30:42

bobbym wrote:

Hi;

Here is my plot of the first 1000 pieces of raw data.

Where do you go from here?

I don't think we're on the same page. Is that the 1st 1000 points or the first 1000 days of ftp://ftp.geolab.nrcan.gc.ca/data/solar_flux/daily_flux_values/fluxtable.txt ? It doesn't look familiar. Remember what I said earlier on about multiple observations per day. The leftmost column "fluxdate" is formatted "YYYYMMDD", e.g. "20041028" is 2004/10/28. I take the average value of all the observations for each day.

My original question was about the significance of the Fourier Analysis when taking a bunch of data and analyzing it as per (replace "fff" with "www" to get past the filter)
fff.brainmapping.org/NITP/PNA/tests/ProblemSet3_files/FourierExcel.htm

I originally fed in the most recent 2048 days. This analysis needs a whole power of 2 as the number of data points.

## #2 Re: This is Cool » sums of power sequences » 2012-05-31 14:32:03

I think I've got a generic format for sums of power sequences. I'm not a math expert, so maybe somebody can express this better. First, is the constraint that
1 = 1, i.e. for any positive integer k

1
∑ i^k = 1
1

n
∑ i^0 = n^1
1

To go to sum of powers of 1...
Step 1) Multiply to clear the denominator of the first term
Step 2) Integrate the expression
Step 3) Add k*n such that (1)=1

n
∑ i^1 = (n^2)/2 + n/2
1

To go to sum of powers of 2...
Step 1) Multiply to clear the denominator of the first term
Step 2) Integrate the expression
Step 3) Add k*n such that (1)=1

n
∑ i^2 = (n^3)/3 + (n^2)/2 + n/6
1

To go to sum of powers of 3...
Step 1) Multiply to clear the denominator of the first term
Step 2) Integrate the expression
Step 3) Add k*n such that (1)=1

n
∑ i^3 = (n^4)/4 + (n^3)/2 + (n^2)/4
1

Rinse; Lather; Repeat. You can take this to any arbitrarily large integer. From this point onwards, it appears that every second co-efficient is zero.

## #3 Re: Help Me ! » Periodicities and Fourier analysis? » 2012-05-31 07:45:52

Is there a way I can send you a spreadsheet or a CSV data file? As I mentioned, there is some cleanup required of the raw data. I've already done that.

Wild hunch time. Is the "7 and 1/2 months" cycle actually 224.7 days? I'm almost hoping that it's not.

## #4 Re: Help Me ! » Periodicities and Fourier analysis? » 2012-05-30 11:14:49

It looks like ftp links aren't filtered. That's the data, which is the important part.

The solarflux data can be pulled from
ftp://ftp.geolab.nrcan.gc.ca/data/solar_flux/daily_flux_values/fluxtable.txt
The column you want is "fluxadjflux"

for earlier data, browse through
ftp://ftp.geolab.nrcan.gc.ca/data/solar_flux/daily_flux_values/

## #5 Re: Help Me ! » Periodicities and Fourier analysis? » 2012-05-29 20:07:30

Sorry, I don't see any links for emailing you directly. Did you mean to cut up the  URLs or leave off the http?

## #6 Re: Help Me ! » Periodicities and Fourier analysis? » 2012-05-29 19:09:55

bobbym wrote:

What I would do is first graph the data. Your own eyes are a great way to spot patterns. If the data is polynomic then a least square square fit is done and an analysis of the residuals. If the dat looks periodic then a Discrete Fourier Fit can be attempted.

Actually, it was after plotting the data that I noticed what look like periodicities. That's what started me looking for an objective analysis, versus simply eyeballing it.

Here's the background... I've been following the 10.7 cm solar flux data from Pentiction. Unlike sunspots, where humans estimate area of sunspots, the 10.7 cm flux is an objective measurement. And furthermore, this measurement is apparently immune to the Livingston-Penn effect but that's a whole other off-topic item.

There are 3 flux readings taken per day, and I run a script under linux that reads the downloaded file, and calculates the mean value for each day. I still have to do some manual cleanup...
1) There are a few 1-day gaps, where the data is missing. I insert the average of the 2 surrounding days.
2) An occasional freak class-X flare will send one observation's value through the roof. I throw that one out, and use the average of the other 2 daily observations.

I plot the data in an Excel-compatable spreadsheet. The most obvious periodicity to look for was approximately 27 or 28 days. That's a "Carrington Rotation" which, in plain English, is how long it takes the sun to complete 1 rotation as viewed from earth. I.e. a noisey sunspot that was pointing at us goes around, and is pointing at us again. That didn't help as much as I had hoped.

Despite how noisey the data is, I noticed pronounced spikes approximately a month and a half apart. 47 days as near as I can tell. Plotting the 47-day running mean greatly smoothed out the data and showed yet another cycle, of approximately 7 and a half months.

To further confuse the issue, the sun was in a solar minimum from mid 2007 through the end of 2009, so the periodicity would be missing during that time.

BTW, I was going to post explanatory links, but apparently new members can't post URLs.

## #7 Help Me ! » Periodicities and Fourier analysis? » 2012-05-29 15:13:45

knighthawk
Replies: 12

Hi all. First post here. I assure you this is not homework. I'm a retiree following my curiousity. Start with 2048 consecutive daily numbers. I ran the Fourier analysis tool in Gnumeric (an Excel clone for linux) and got 2 columns (real+imaginary) of 2048 numbers each.

The program manual told me how to run the analysis. The next question is what do the output numbers mean?

Another option... is there a canned program somewhere (free; runs under linux) that can find periodicities in data sets?