New post
1

SD Card Full - Old Cached MRSS files not being deleted.

Avatar
JRB Technical
None
Follow

I thought that the BrightSign units were supposed to be deleting old cached content when the SD card runs out of room?

I just had another player (HD220) stop displaying MRSS content, no error lights or anything to indicate a problem. I tried rebooting the player, player came up fine, but still would not play content, checked network connection, etc. all the things to troubleshoot normally.

Pulled out the SD card (16GB), to find that it was completely full.

Deleted all the files in the Cached directory, and reinstalled the card, rebooted the player, and all is well again.

What is the mechanism for the players to delete cached files?
Delete older unused cache files after X many days of use? 
Delete older unused cache files after card is X% full? 
How is this supposed to work?
Why is there no error light indications for a full SD card?

Can this be addressed to do a better job than it currently seems to be?  

I would not think that I have that aggressive of content on this player, but it is a pain when a 16GB card fills up in 30 days, when there is only simple content on the player.

Please let me know, so I can best advise my clients who are also having the same issue.

Thanks.

John

Date Votes

72 comments

  • 0
    Avatar
    Lyndon

    There was a bug with older brightauthor release where the cache wasn't getting cleaned properly, but after that a hard cache clear was added at 4000 files until a true fix was implemented. Since BrightAUthor 3.4, though, the cache should be getting cleared properly.

    So, unless you're using an older brightauthor release the cache should be cleared automatically once it reaches a percentage of the sd card availability. 

  • 0
    Avatar
    JRB Technical

    Thanks!

    I pass on to my clients.  Unfortunately everyone is running different versions of Firmware and BrightAuthor, based on their use and needs.  Many have had issues with some items in the more recent versions, and are now reluctant to upgrade for fear of more problems being introduced, until there is time proven stability with the newer releases.

  • 0
    Avatar
    Lyndon

    If you let me know what brightauthor versions are seeing this, I can look into a manual purge after x number of files are in the cache. 

  • 0
    Avatar
    JRB Technical

    There are really so many different BA versions being used by people utilizing my Media RSS feeds right now, to try and implement any solution other than notifying them that this may be an issue with some versions, and to periodically check the SD cards to see if they are approaching capacity.

    Thanks for the offer though.

  • 0
    Avatar
    JRB Technical

    Can I please get some confirmation from the BrightSign staff, as to which BrightAuthor versions and firmware versions the cache issue is supposed to be fixed in?

    I have not tired the absolute latest releases yet, but I am hearing from some of my clients that there still seems to be issues in versions since BA 3.4 filling up the SD card completely.

    Also, what exactly are the parameters in which the player is supposed to delete older cached files?  If it is a percentage of the SD card memory, what is the exact percentage?  Does it do this immediately when it reaches that percentage? Or once a day, or once a week, etc.?

    I need to be able to test and confirm a solution that works, before having people spend the time upgrading a large number of players again, when the problem doesn't seem to be solved.

    Thank you for any additional information you can provide on this issue.

    John

  • 0
    Avatar
    Alex K.

    MRSS playlist logic (BrightAuthor 3.5.0.24):

        1. Unit downloads the feed.
        2. Sets a timer based on the TTL in the stream
        3. Starts to download and play the items in the stream
        4. Continues downloading and playing until TTL timer expires.
        5. After the TTL timer expires it download the feed again.
        6. It then waits until the end of the previous feed to switch to the new feed and goes to step 2

        How MRSS manages storage (How MRSS Caching works):

        For Images:
        The cache will allow as many images to collect up to 100MB left on the card or 4000 files. Then all in cache that aren't in the current feed are deleted.

        For Videos:
        If the size of the video is specified in the RSS feed, they will only be deleted when space is required oldest first.
        If the size of the video is not specified in the RSS feed, they will be cached up to 100MB left on the card, then all not in current feed will be deleted.

        The card MUST be large enough to store all the assets in a feed. If you have a 32MB card and a feed with 1000 images totaling 300MB it will not work.

        There is a setting in autorun script that determines how much free space triggers clearing of the cache. This is the line that sets the 100MB value that's referred to:
        rv.reservedAmountOnCard = 100 * 1024 *1024 'reserve 100 MB on card

  • 0
    Avatar
    JRB Technical

    OK, so it isn't a percentage, but when the card gets down to 100MB.

    In the case of my typical Weather Feed for clients:

    US Radar Image @400KB (TTL=5) new image every 5 minutes for 288 images a day - 115.2MB
    Current Conditions @400KB (TTL=60) new image every hour for 24 images a day - 9.6MB
    Forecast @400KB (TTL=60) new image every hour for 24 images a day - 9.6MB
    (+ NWS Alerts @200KB unknown number of images based on severe weather)
    Comes to about 135MB a day for 30 days or about 4.05GB a month.

    So - assuming a 4GB card, the card is going to fill up in about 1 month or less, and then sit and fight over the the last 100MB on the card all the time, when all of those cached images have not been used in quite a while and are just taking up space on the card.

    Sounds like the current logic for cached files perhaps needs to be rethinked to something different, maybe along the lines of a cached file hasn't been accessed in 7 days, so it can be deleted.

    I will do some tests with changing the rv.reservedAmountOnCard to a higher value and see if that helps any.

    John

  • 0
    Avatar
    JRB Technical

    Ok

    I did a test on a 4GB card starting 2 days ago on my demo system that monitors many different weather feeds, so the cache builds up very quickly

    I changed the rv.reservedAmountOnCard to the following:

    rv.reservedAmountOnCard = 3500 * 1024 *1024 'reserve 3500 MB on card

    I let the player run for a couple of days, I checked the card just now, and free space on the card was only 2.39GB, and if I understand correctly, the free space should have been closer to 3.5GB (The project files on the card are only about 2MB total).

    This was on a HD220 with Firmware 4.4.44 and using BA 3.5.0.24.

    I have also heard back from another client using BA 3.5.0.24 who has not changed the default  rv.reservedAmountOnCard but has said that they are still seeing issues with the SD card becoming completely full with that version of BA.

    We really need to have a solution that works for automatically deleting cached files, without having to make any changes to the autorun script, and without having to go in and manually delete the cache folder all the time.

    The whole point of using Media RSS is to display content that changes every day, every hour, every 5 minutes, etc.  The cache is going to build up quickly, so this really needs to be addressed please.

    Thanks.

    John

  • 0
    Avatar
    Lyndon

    There's a bug request in to fix this issue.  Until that fix is in place, we'll need to modify the autorun file to handle it properly. The recommended change should have capped how much space the mrss feed could use. I'll do some additionally testing to see if there's another location where that value is set or being overwritten. 

    Please confirm that the autorun file on your sd card is the one you modified.

  • 0
    Avatar
    JRB Technical

    I'm using Simple File Network, used custom autorun for project with the change, verified that  the change was in the autorun file on the server as well as on the card when I started the test, and again just now for all locations.

    John

  • 0
    Avatar
    JRB Technical

    Also, just to be clear, that value change was the only modification to the custom autorun from the standard autorun.

    John

  • 0
    Avatar
    Lyndon

    I found the problem with the previous change. THe reserved space value is only used when file sizes are specified in the feed. I'm chaning that logic, and will send the updated change to make. 

  • 0
    Avatar
    JRB Technical

    Any more news on this, a temporary fix, and or when this fix may be in the the next published version?

    Thanks!

    John

  • 0
    Avatar
    JRB Technical

    Hello, anyone there?

    Is there any more news on this?  Availability of temp fix?? Estimated release date of permanent fix???

    I have a client with a large number of BrightSigns spread out over a large facility, and this has become a huge headache for them.

    I would like to be able to test and provide them with a temporary solution to this, before the permanent fix release if at a possible.

    Thanks!

    John

  • 0
    Avatar
    Lyndon

    I should have something new for you to test tomorrow. 

  • 0
    Avatar
    JRB Technical

    Any more news on this?

    Is this fixed in the new Beta Versions that were released April 11th (I see no mention in the Release Notes)?

    John

  • 0
    Avatar
    J Fleck

    Is there any new news?  Also, even a quick script that will clear the files, but will startup the unit so I can remotely load the basic setup for the device.

  • 0
    Avatar
    JRB Technical

    Hello?

    Is anyone from the BrightSign Support Staff going to respond to this???

    I have a client that is trying to close out an install of 20 BrightSign players with this issue right now, and they can't until this issue is resolved.

    I also have another client with 6 players that also needs a working solution.

    Please, I really need to get a working solution to this issue that I can test and confirm as a working solution ASAP.

    Thank you.

    John

  • 0
    Avatar
    Lyndon

    We're working on a temporary workaround for current brightauthor releases. I'll post changes to this thread this week. 

  • 0
    Avatar
    Lyndon

    There are two lines to edit to force the brightsign to adhere to the reserved space limit. 

    rv.reservedAmountOnCard = 100 * 1024 *1024 'reserve 100 MB on card

    and 

    if (sdfreeSpace < 100) or forceClean then

     

    So, to set brightsign to stop the cache from growing if there's less than 1GB of free space on the sd card, you would change the lines to the following:

     

    rv.reservedAmountOnCard = 1000 * 1024 *1024 'reserve 100 MB on card

    and 

    if (sdfreeSpace < 1000) or forceClean then

     

    I'm attaching a modified version of the autorun file for brightauthor version 3.5.0.24 and 3.5.0.31. 

  • 0
    Avatar
    Lyndon

    3.5.0.24 and 3.5.0.31 autorun versions.

  • 0
    Avatar
    JRB Technical

    Ok - I tried testing on a 16GB card that was about 54% full (7290 MB free space) with cached Media RSS image content, and I changed the "1000" in both lines to "8000"  under the assumption that if I am reading correctly, that should delete cached content if there is less than 8GB in free space available on the card.

    I rebooted the player, and it started by not displaying any of the Media RSS images during the first pass (it normally displays current cache images after rebooting with cache content still intact), and did not seem to delete any cached files.  When it got back to the beginning of the Playlist loop, it started displaying Media RSS images again.   It just does not seem to be doing anything about deleting cached content to make more available space for new content. The amount of free space on the card remains unchanged.  I have also tried using "9000" MB, with the same results.

    I will try again on a smaller card with the autorun.brs at 1000 from the start, to see if there is any difference, but I don't see any reason why it shouldn't have worked with the larger numbers for testing. Unfortunately it will take a few days to build up a cache large enough to trigger the delete cache routine.  I am not sure what else I would need to change if anything to set other arbitrary free space values for testing purposes to verify that this is working.

    John

  • 0
    Avatar
    Lyndon

    I'll test it with a larger cache setting. I tested this and a colleague of mine did a similar test set to 3500. I tested it set a little under 1GB. In both cases, the unit would clear room to make sure the cache didn't get to large. 

  • 0
    Avatar
    Mkeen

    Hi Brightsign,

    I am one of John's clients who is experiencing this problem.  We have 4gb SD cards and they fill up in about 3 weeks.  I'm glad to see that you guys are working on this as it's very frustrating.

    One thing that would help us in the interim while you work this out: It would be very nice if we could use the diagnostic web server interface to clear the cache manually.  Currently, it allows us to delete only one file at a time which means it would take about a half an hour of mouse clicks on EACH player to clear the cache.  Do you think you could add a way to delete an entire directory at once?

    Thanks.

    Michael Keen

    Southern Vermont College

  • 0
    Avatar
    JRB Technical

    Ok, the player I had running from a fresh start on a 4GB card with the autorun.brs modified for a 1GB reserved space has reached that limit.

    It is now past that limit:

    Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mmcblk0p1         3805184   2936544    868640  77% /storage/sd

     

    When it reached (and passed) that limit, no files are/were deleted.  Also, these modifications seem to bog down the player when it reaches the limits, in a manor in which image files that are set to be displayed for 10 seconds, are being displayed for around 30-40 seconds (variable from image to image).

    This is the same behavior I saw with the larger SD card that was already mostly full, in which I set a larger limit than the 1GB for testing purposes.

    This does not seem to solve the problem at all with deleting outdated cached images.

    My customers keep asking me what the status is on this, as this thread was started over 3 months ago now.  I had told them that I was waiting on a solution from BrightSign. And we are all still waiting, as patiently as possible under the circumstances.

    Can we please get some working solution to this issues as soon as possible?

    John

  • 0
    Avatar
    Lyndon

    I'll try and duplicate the failure you're seeing. John, can you send me a sample project, just the brightauthor bpf file. You can email it to support.

    Michael, I'll check on the diagnostic server. 

  • 0
    Avatar
    Lyndon

    John, can you send me the autorun.brs file on one of the sd cards where the modifications I sent aren't working?

  • 0
    Avatar
    JRB Technical

    I already sent you that file a few days ago, along with the project file requested.

    Please read my message that I sent with the files I sent on May 7th.

    Ticket #14317

    John

  • 0
    Avatar
    Kevin Meyer

    We're watching this thread as we've experienced similar issues with presentations that include MRSS feeds, ultimately leading to dead screens with "SD card locked" or other misleading error messages on the screen.  I've replaced SD cards in some cases and the cards that have been returned from the field are indeed full.  The preventive measure I've found successful over time...pending a permanent fix...is to send routine weekly Recovery - Reformat commands.  It's monotonous but manageable so far...obviously something to do after business hours, usually from my living room on Sunday nights.  But we're blessed with a growing list of end users and definitely looking forward to the fix.  Anything new?

  • 0
    Avatar
    Mkeen

    Kevin, please elaborate on this Recovery - Reformat procedure. How are you doing this remotely?

Please sign in to leave a comment.