Matt Murphy on 28 Mar 2016 08:18:43 -0700
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
[PLUG] Odd script problem
- From: Matt Murphy <email@example.com>
- To: "Philadelphia Linux User's Group Discussion List" <firstname.lastname@example.org>
- Subject: [PLUG] Odd script problem
- Date: Mon, 28 Mar 2016 11:18:34 -0400
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to; bh=u45GnXBbiFsw85V7jnfLijZ2gn4+GXDQPDkgXSc43zk=; b=CazMOSanaG/H5ye2zqikcKJa8FH+kh+hxzCVWetDmWEFmtk7GS3Zdr2IHCywbhLWFz SCdZcKAQj/l9rtSv+tHYaGfMvkCcpfPzdI67DXfch6HTaYshuZMB+K3sLUehSktFhRdt aAvcYeztD3ZV1W2XO3VQWe8MJjlJqpPmi0QV2fTlXCc2FW8qlBpGFClu5boAT0SMliHa 9gMaOHMU3dES8Wiaw2QzaH64ADbEUn5WjdcDCL6A+kg6VYaEnk3385drnZaoVeMu5mT/ 218FzX6t7AjSZlEBUdAwbCkroo//exavPTjc+ypJFWEMnyBim+B+lYwlC28mmVuhR7ID j2lA==
- Reply-to: Philadelphia Linux User's Group Discussion List <email@example.com>
- Sender: "plug" <firstname.lastname@example.org>
I'm a long-time linux user and frequent defacto admin.
I have a problem: a cron job is running that calls a bash script to change the directory and run a php script which, in turn, scrapes an old-and-ugly site which our office owns but does not control (meaning that doing something sensible like serving the data we now scrape via an API is not an option). With the size of the content currently being scraped, the run time is one or 2 seconds. The cron job runs, at peak, once a minute.
Naturally, I do some logging: The bash script writes out a timestamp to log when it runs, the PHP script writes 'locked' to a status file and 'working' and a timestamp to log. Any exceptions are captured and logged out with timestamp when they occur. End-of run is also logged, with timestamp and, finally, 'unlocked' is written to the status file.
The problem: I'm showing some instances when the cron job runs and the php job doesn't, at all (no status written, no 'working' log entry). It's not being caused by collision (recall the scrape runs in under 2 seconds and the cron interval is 1m and there's app-level logging-o-plenty to expose a collision); additionally, I've seen similar at longer increments (2, 5 mins). PHP error level is set to E_ALL and there is no output that corresponds to the missed runs.
My thinking is that I might be looking at a failure of one scripting environment to hand off to another.
Has anyone seen something like this? I'm having trouble finding system log activity that explains what I'm seeing.
Philadelphia Elections Commission
Philadelphia Linux Users Group -- http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion -- http://lists.phillylinux.org/mailman/listinfo/plug