[Aide] AIDE performance problems

[Paul Hessels]

I will give that a try although its far from ideal.

Not that it likely matters, but I mis-read the number of files.  Its only
20 million.

On Sun, Apr 5, 2015 at 7:51 PM, Brian Mathis <
brian.mathis+aide at betteradmin.com> wrote:

> Try to split the run across multiple processes by making different
> aide.conf files that covers different sets of files, then start each
> process with "-c config.conf" and see what happens.  If it goes faster,
> then you're CPU bound.  If not, then it's IO.
>
>
> ❧ Brian Mathis
> @orev
>
>
> On Sun, Apr 5, 2015 at 5:26 PM, Paul Hessels <paul at hessels.ca> wrote:
>
>> I failed to mention, I am only using md5 as it stands.
>>
>> On Sun, Apr 5, 2015 at 5:50 PM, Andy Lawrence <dr.diesel at gmail.com>
>> wrote:
>>
>>> Are you concerned with intrusion detection or only mystery bit flips?
>>> If only bit flips change your hash on that directory to only md5, will be
>>> much faster.
>>>
>>> On Sun, Apr 5, 2015 at 5:11 PM, Paul Hessels <paul at hessels.ca> wrote:
>>>
>>>> I have a rather large dataset that I am running AIDE against. I am
>>>> running AIDE on my backup server that has a bunch of systems backups on
>>>> it.  The server in question has about 5TB of data and about 200 million
>>>> files.
>>>>
>>>> Currently its taking about 27 hours to run.  The system is 15 drives in
>>>> raid0 with XFS.
>>>>
>>>> Oddly, the bottle neck doesn't seem to be disk.  iostat lists it as
>>>> only 25% util.  Simple dd tests suggest that percentage is right for both
>>>> throughput and iops.
>>>>
>>>> The CPU doesn't seem to be the bottleneck.  Its about 65% idle most of
>>>> the time.  The process seems to be sitting in the 'D' state most of the
>>>> time.
>>>>
>>>> The number of interrupts seems reasonable...
>>>>
>>>> I don't know what to do next.  I need this to run in 20 hours or less.
>>>>
>>>> _______________________________________________
>>>> Aide mailing list
>>>> Aide at cs.tut.fi
>>>> https://mailman.cs.tut.fi/mailman/listinfo/aide
>>>>
>>>>
>>>
>>>
>>> --
>>> projecthuh.com
>>> Never underestimate the carelessness of boredom...
>>> Most people prefer Windows because most people are idiots...
>>>
>>>
>>> _______________________________________________
>>> Aide mailing list
>>> Aide at cs.tut.fi
>>> https://mailman.cs.tut.fi/mailman/listinfo/aide
>>>
>>>
>>
>> _______________________________________________
>> Aide mailing list
>> Aide at cs.tut.fi
>> https://mailman.cs.tut.fi/mailman/listinfo/aide
>>
>>
>
> _______________________________________________
> Aide mailing list
> Aide at cs.tut.fi
> https://mailman.cs.tut.fi/mailman/listinfo/aide
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cs.tut.fi/pipermail/aide/attachments/20150405/6481d2d5/attachment.html>