Database Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Codewalkers ForumsOther TechnologiesDatabase Help

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Codewalkers Forums Sponsor:
  #1  
Old August 11th, 2009, 07:51 AM
Ashberry Ashberry is offline
Registered User
Codewalkers Newbie (0 - 499 posts)
 
Join Date: Aug 2009
Posts: 1 Ashberry User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 28 m 34 sec
Reputation Power: 0
[MySQL 5] Finding and deleting duplicate entries in a large database table

Hi everyone!

I have the following problem. A large database which through some error, now contains a high number of duplicate entries within one of the columns. Now normally I would simply do things by hand, deleting the duplicate entries myself. However in this particular case, the database contains thousands of entries within this one table alone. Far more than I could ever possibly fix by hand.

So I was wondering if there is a query, or method or piece of software which could be used to find and select just those rows which have duplicates within one of the columns, and if possible, delete one row of each duplicate (doesn't matter which is saved and which is removed as both are exactly the same, so long as one of the copies remains intact).

The following may help

Primary key = id
Object_id = numerical (will be duplicated also in many cases)
X_position = numerical (this is the one containing duplicates)
Z_position = numerical
Y_position = numerical

I'm still pretty new at the database thing, and ended up having to do this because the person responsible kept passing the buck. I'm not afraid to get my hands dirty and delving in there, because I find all this really interesting. But I am a little out of my depth when it comes to something like this, and I don't entirely know what I'm doing. But as it does need doing, I'm hoping someone can help me out, and I didn't really know where else to ask.

To further explain what's happening. The table is used to populate a map with instances of various 3D objects (Object_id), and works as intended. However the duplicates are causing second (and in some cases third) copies of the same instance to be placed at exactly the same location as many of the placed objects, with only very slight differences in orientation and positioning on the Z and Y axis', causing lots of clipping errors, extra work on the engine and visually it looks terrible. It's unprofessional and can't be left like that.

Last edited by Ashberry : August 11th, 2009 at 08:49 AM.

Reply With Quote
  #2  
Old August 28th, 2009, 05:33 PM
lig's Avatar
lig lig is offline
"Forum Nazi"
Codewalkers Demi-God (4500 - 4999 posts)
 
Join Date: Apr 2007
Location: Jacksonville, Fl
Posts: 4,753 lig User rank is Private First Class (20 - 50 Reputation Level)lig User rank is Private First Class (20 - 50 Reputation Level) 
Time spent in forums: 4 Days 8 h 15 m 54 sec
Reputation Power: 7
Assuming this is MySQL

Hmm - maybe to find out what values have more then one entry - use a count with a group by
Code:
SELECT value, if(count(*) > 1, 'Yes', '') FROM table GROUP BY value

From there you can delete by hand or write a script to do it for you based on the results - maybe with a stored procedure or the lang of your choice

Make sense?
__________________
life is a game.... Have fun
-----------------------------
http://www.phpwomen.org
strength in unity

PHPCommunity IRC
#phpc on freenode

Reply With Quote
Reply

Viewing: Codewalkers ForumsOther TechnologiesDatabase Help > Finding and deleting duplicate entries in a large database table


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump




 Free IT White Papers!
 
How to Present Effectively Online
This white paper offers practical and actionable advice on the key steps that any presenter should consider as they plan and execute a Webinar or online meeting.

Request Your Free Technology Downloads!
 
Open Source Security Myths
Open Source Software (OSS) is computer software whose source code is available to the general public with relaxed or non-existent intellectual property restrictions (or arrangement such as the public domain), and is usually developed with the input of many contributors.

Request Your Free Technology Downloads!
 
Power and Cooling Capacity Management for Data Centers
This paper describes the principles for achieving power and cooling capacity management.

Request Your Free Technology Downloads!
 
Scalable, Fault-Tolerant NAS for Oracle - The Next Generation
For several years NAS has been evolving as a storage alternative for Oracle databases, and for good reason: NAS is quite often the simplest, most cost-effective storage approach for Oracle. Learn about the benefits that HP's approach to scalable NAS brings to Oracle environments in this comprehensive white paper.

Request Your Free Technology Downloads!
 
Understanding Web Application Security Challenges
This white paper discusses many common threats and preventive measures for Web application security, and explains what you can do to help protect your organization.

Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 




© 2003-2009 by Developer Shed. All rights reserved. DS Cluster 6 Hosted by Hostway
For more Enterprise Application Development news, visit eWeek