Introduction

This wiki page presents some results on bulk lfc methods.

Bulk methods which have been tested

There are four methods.

• Test 3: int lfc_delfilesbyguid (int nbguids, const char * guids, int force, int * nbstatuses, int * statuses) : allows deleting the replicas and LFNs associated with the given list of guids if the flag force = 1. if force = 0 and there are still replicas for a given guid, nothing is deleted.
• Test 4: int lfc_delfilesbyname (int nbfiles, const char * paths, int force, int *nbstatuses, int *statuses) : allows deleting the replicas and LFNs associated with the given list of LFNs if the flag force = 1. if force = 0 and there are still replicas for a given LFN, nothing is deleted.
• Test 5: int lfc_delfilesbypattern (const char *path, const char *pattern, int force, int *nbstatuses, Cns_filestatus **statuses) : allows deleting the replicas and LFNs if the LFN is like 'path/pattern' and the flag force = 1. if force = 0 and there are still replicas for a given LFN which matches the pattern, nothing is deleted.
• Test 6: int lfc_delreplicas (int nbguids, const char * guids, char * se, int * nbstatuses, int * statuses) allows deleting the replicas only.

Local LFC server and client

Hw and nw properties

• LFC server :lxb0981.cern.ch with MySQL (SL4)
• LFC client : lxb0728.cern.ch (SL3)
• ping time 0.34ms
• nb of entries in the DB : 1349134

Test conditions

Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).

• Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
• Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
• Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
• Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
• Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
• Test 6 : lfc_delreplicas to delete replicas only.

Test results

Execution time is given in second.

Type of test \ #files to delete	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6
0	0.32	0.32	0.33	0.37	12.92	0.32
1	0.68	0.35	0.35	0.38	13.41	0.33
10	6.56	0.55	0.44	0.52	12.59	0.41
100	66.94	2.18	1.07	1.19	13.07	0.83
500	335.92	8.39	4.16	4.64	15.89	3.85
1000	677.16	24.65	9.04	9.30	24.78	7.68
5000	3344.46	96.79	56.27	62.05	71.56	34.75
10000	6730.47	194.48	121.54	120.45	128.22	68.76

Execution time of the different type of test in a graph.

This table gives the execution time to delete one file (one LFN + one replica, except for Test 6 where there is only one replica deleted, no LFN). Execution time is given in second.

Type of test \ #files to delete	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6
0	0.320	0.320	0.330	0.370	12.920	0.320
1	0.680	0.350	0.350	0.380	13.410	0.330
10	0.656	0.055	0.044	0.052	0.126	0.041
100	0.669	0.022	0.011	0.012	0.131	0.008
500	0.672	0.017	0.008	0.009	0.032	0.008
1000	0.677	0.025	0.009	0.009	0.025	0.008
5000	0.669	0.019	0.011	0.012	0.014	0.007
10000	0.673	0.019	0.012	0.012	0.013	0.007

Local LFC server and remote LFC client

Hw and nw properties

• LFC server :lxb0981.cern.ch with MySQL (SL4)
• LFC client (Tawain) : lcg00122.grid.sinica.edu (SL3)
• ping time : 312.64 ms
• nb of entries in the DB : 1349134

Test conditions

Tests on different ways to delete files in the LFC. Each test has been repeated three times (same number of entries in the DB).

• Test 1 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs.
• Test 2 : lfc_delreplica and lfc_unlink to delete both replicas and LFNs inside a session.
• Test 3 : lfc_delfilesbyguid to delete both replicas and LFNs (force = 1).
• Test 4 : lfc_delfilesbyname to delete both replicas and LFNs (force = 1).
• Test 5 : lfc_delfilesbypattern to delete both replicas and LFNs (force = 1).
• Test 6 : lfc_delreplicas to delete replicas only.

Test results

Execution time is given in second.

Type of test \ #files to delete	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6
0	3.06	3.68	3.06	3.21	15.37	3.07
1	6.19	4.03	3.08	3.23	15.88	3.07
10	63.30	9.84	3.19	3.38	15.58	3.41
100	626.30	71.22	4.09	4.52	16.31	3.89
500	3182.93	331.01	8.09	8.31	19.26	7.90
1000	6233.26	654.35	12.25	12.94	23.08	11.28
5000		3294.76	61.30	68.45	72.01	42.23
10000		6534.10	125.96	131.92	133.49	78.75

Execution time of the different type of test in a graph.

Execution time is given in second.

Type of test \ #files to delete	Test 1	Test 2	Test 3	Test 4	Test 5	Test 6
0	3.060	3.680	3.060	3.210	15.370	3.070
1	6.190	4.030	3.080	3.230	15.880	3.070
10	6.330	0.984	0.319	0.338	0.156	0.341
100	6.263	0.712	0.041	0.042	0.163	0.039
500	6.365	0.662	0.016	0.017	0.039	0.016
1000	6.233	0.654	0.012	0.013	0.023	0.011
5000		0.651	0.012	0.014	0.014	0.008
10000		0.653	0.0126	0.013	0.0133	0.008

-- LanaAbadie - 27 Nov 2007