database - telling if key exists in mysql table is taking too long -


i started question: is large mysql table destined failure?

the answer found question satisfactory. have table 22 million rows grow 100 million. @ time, table minute_data structure this: enter image description here

a problem having follows. need execute query:

select datediff(date,now()) minute_data  symbol = "csco" order date desc limit 1; 

which fast ( < 1 sec ) when table contains value "csco". problem is, query symbol not in table already. when execute query for, say, symbol = "abcd":

select datediff(date,now()) minute_data  symbol = "abcd" order date desc limit 1; 

then query takes long time... forever ( 180 seconds ).

a way can around making sure table contains symbol looking before execute query. fastest way found follow query, need use check see if table minute_data contains symbol looking or not. need return boolean value know if symbol in table or not:

select count(1) minute_data symbol = "csco"; 

this query takes on 30 seconds return 1 value, way long liking, since query above, returns datediff calculation takes less 1 second.

symbol column part of pri key, thought should able figure out if value exists there quickly.

what doing wrong? there fast way want do? should change structure of data optimize performance?

thank you!

update

i think found solution problem. answer below lastcoder, did following:

1) created new table called minute_data_2 exact same definition minute_data.

2) alter table minute_data_2 add primary key (symbol, date);

3) insert ignore minute_data_2 select * minute_data;

4) drop table minute_data;

5) rename minute_data_2 minute_data

now seeing blindingly fast speed same query described above taking more 180 second, completes in .001 seconds. amazing.

did try using exists (...)

select datediff(date,now()) minute_data   exists(select * minute_data symbol = "csco")  , symbol = "csco" order date desc limit 1; 

even though symbol primary key, seems have timestamp pk makes me think using composite pk means ordering timestamp symbol. may want put separate index on symbol, if have composite 1 timestamp first.


Comments

Popular posts from this blog

php - Wordpress website dashboard page or post editor content is not showing but front end data is showing properly -

How to get the ip address of VM and use it to configure SSH connection dynamically in Ansible -

javascript - Get parameter of GET request -