What is data mining?

[razorman]

I have been hearing about "data mining" lately, in relation to the "Able Danger" situation, and adware. I am not sure what it means. Could someone here explain it to me? Thanks.

[RobertSeattle]

1. from Wikipedia

http://en.wikipedia.org/wiki/Data_mining

Definition
Data mining has been defined as "The nontrivial extraction of implicit, previously unknown, and potentially useful information from data" 1 and "The science of extracting useful information from large data sets or databases" 2. Although it is usually used in relation to analysis of data, data mining, like artificial intelligence, is an umbrella term and is used with varied meaning in a wide range of contexts.

A simple example of data mining is its use in a retail sales department. If a store tracks the purchases of a customer and notices that a customer buys a lot of silk shirts, the data mining system will make a correlation between that customer and silk shirts. The sales department will look at that information and may begin direct mail marketing of silk shirts to that customer, or it may alternatively attempt to get the customer to buy a wider range of products. In this case, the data mining system used by the retail store discovered new information about the customer that was previously unknown to the company.

[the_spectator]

2. No expert here, however

it does basically seem to mean "finding useful information" or "analyzing the information you have" etc. etc.

It seems to be just another fancy corporate-jargon fad.
Kinda like "going forward"?

[razorman]

3. I think you are probably right.

I had never heard the term until a week or so ago. Since then, it seems to keep cropping up.

[ProfessorGAC]

14. Nope! Not A New Buzzword

It's been in use by statistical analysts for MANY years. It's been adopted by the Six Sigma movement, but actuaries, statisticians and scientists have been using the term data mining for as long as i've been in the field. (I was an undergrad in the early to mid-70's.)

It's not anything new and is really a good way to get those new to the field to understand the concept. Simply setting the data in a different arrays and dimensions to determine the interactions and causative links provable by the math.

I've been using that term for at least 30 years.
The Professor

[razorman]

18. That's good to know.

But why do you suppose it seems to be used more publicly at the moment?

[Rich Hunt]

29. probably because

It's probably because the technology has improved and expanded so much in the past few years - I'd say the past four to five years especially have created greater opportunities for "cross-referencing" data and drawing up "profiles".

If you work with computers all day - especially in any sort of job that requires gathering information, you are familiar with what I'm talking about.

This reminds me of the b.s. trendy job title of "information broker". I actually saw someone on an industry mailing list use this self-description and nearly choked.

[Perky]

4. Data MIning

Basicallyu it is taking data from a variety of sources and finding links between them. SOme time int he case of adware is it done at the macro level to extract or profile a specif type of consumer. they then take that profile and span everyone who meets it. The volumes are usch that while they have indicidual eamil addreses of those who meeet the criteris, no one really cares.

Data MIni can also be done to chross match mailing list,
website visits,,airlione flights, credit card purchases, phone calls, etc top isolate individuals who might be up to no good. Its a bit scary whaen you think about how it can be abused, but it can also be used to find your Mohammed Atta's

[razorman]

10. Like most technology...

I suppose that it can be great when used properly, but can also be easily misused, much like the internet itself.

[Ready4Change]

5. Different things to different people.

There are times when you have too much data. So much that it's hard to tell what it tells you.

Legitimate analysts will pour over the data, finding various trends, relating them to each other, finding out if there are underlaying currents which can explain perplexing phenomena.

OR, you can delve into the data to find ways of supporting YOUR point of view.

[jody]

6. The various efforts to reveal the Bible Code is an instance of Data Mining

Data mining includes methods that link apparently disparate data bases so they can be used to answer higher order questions. It's also a front for BS research.

[razorman]

12. I think I see how it could be useful for BS.

For instance, how would you disprove someone's BS theory obtained through data mining?

[jody]

15. You broke the code.

:toast:

[mcscajun]

7. In relation to adware and spyware...

...Data mining takes advantage of information gathered by a Web site and collects such information, often without the user's knowledge.

So, when your anti-spyware or anti-adware programs identify a threat from a data miner, the threat is to your privacy.

[razorman]

13. I just got rid of some of those files using Ad-Aware today.

That's what prompted me to ask about this.

[EC]

8. lists ... n/t

[GreenPartyVoter]

9. Welcome to DU

[razorman]

16. Thank you.

I appreciate it. I have been visiting DU for several weeks now, and enjoy reading many of the discussions.

[GreenPartyVoter]

19. It's a real interesting place. For my part, I can't stay away for very

long at a time. *g*

[razorman]

21. I understand that.

Although I am not really sure if I fit in here. I am not as liberal as most DUers, apparently. However, I like to hear all sides of the issues. I find that I can often learn something from my friends who are more liberal than I am, as well as the ones who are more conservative. What I don't like to see is the amount of vitriol that appears on both ends of the spectrum. I have strong opinions of my own, but I know no one will listen to them if I am screaming.

[GreenPartyVoter]

22. I hear what you are saying. It's what one of the reasons why I support

multi-party politics. I just can't believe that 2 parties can cover every one. :)

[razorman]

24. I agree.

It is especially true when the two parties only yell at each other and accuse each other of all sorts of nefarious deeds and motives.
I enjoy a good civil discussion with someone who has a different opinion than mine. Life is too short to stay pissed off all the time.

[GreenPartyVoter]

26. Yep. :^) If you have time check out my list of

election reforms (down in my sigline). A lot of them would help to change the incivility of political discourse, at least among candidates. (For one thing, if you have more candidates they just don't have time to slime and sling mud like they do when there is only one major contender.)

[razorman]

27. How do I access that?

[GreenPartyVoter]

28. Here you go. You'll have to scroll down a bit

http://timeforachange.bluelemur.com/electionreform.htm

[Alcibiades]

11. In ant large dataset

there are bound to be things that are correlated with one another. One of the problems with data mining is that it can turn up correlations that are completely accidental.

[yy4me]

17. DataMiner

When I do scans for adware and spyware in conjunction with my anti-virus scans, Ad-Aware always finds dataMiner as one of the tracking programs that have latched onto my computer. There are several web sites that I visit often. One of them in particular tracks me, I'm sure I know which one it is that has latched onto me to see what sites I visit. I hope you have some good anti-virus and anti-spyware and adware programs or you will be watched and tracked no matter where you go on the net. I am fairly new on the computer but have learned how to do the scans and I downloaded them FREE from a trusted web site on the advice of a computer expert. If my terminology is off a little, please forgive this novice.

[razorman]

20. I use Norton and Ad-Aware

They seem to work pretty well for me. If not, maybe I can just dance around my desktop and pray to the Great Computer God to protect me. I'm not sure anything really works. lol.

[yy4me]

23. Hope this is not considered an ad but

I got all my anti-virus goodies, including (bless them) a pop-up blocker from MajorGeeks.com. All kinds of goodies. I use Ad-Aware,
Spybot-S&D, Anti-Vir and SpywareBlaster. Never a problem.

[razorman]

25. Wow. With all that protective software...

I am afraid that I wouldn't have room for much else on my hard drive. lol.

[Rich Hunt]

30. screw that

If people want information on you, there are lots of other ways to get it. They'll get it one way or another if they want it bad enough.

I'm not going to take the bait and spend a bunch of money on crap that won't protect me. I'm not gonna shill for companies, either, so I'll leave it at that.

[yy4me]

31. Sorry Rich Hunt

No harm was meant, I was only trying to pass on some information that I learned after having to spend a great deal of money having my computer fixed. It is not my position to shill for anyone(and it is all free). I see by your number of posts that you are a long and faithful participant on DU. Please give us new folks a chance to see how things work. These are harried times, It was not my intention to ruffle your feathers. This has happened to me twice in 2 days. I may reconsider what I am here for. I thought it was for the exchange of ideas.