Possible PIC port problem with linker script.

Using the freeRTOS PIC18 port. MPLAB C18 memory model set for: Small code model, large data model, multi-bank model. I.e. as per freeRTOS demos. I have discovered what I think is a problem with the linker script used with the PIC port of freeRTOS.  This only showed up recently, despite the fact that I have been using freeRTOS for many weeks.  I discovered that by declaring an arbitrary  char array of a particular  length,  I could make my application fail to run correctly. Investigation revealed that variables defined in tasks.c were not being correctly addressed by the code generated by the  C18 compiler. This was only a problem if  task.c defined variables crossed a bank boundary, hence the dependence on the size of my char array. The RAM data defined in a C source file is placed in a section specific to the source file. I.e. the RAM data defined in tasks.c is placed in either idata_tasks.o  or  udata_tasks.o  The linker will place this in one of the data memory regions defined in the linker script. The linker ensures that a section will never cross a region boundary, they will always be placed in just one region. However, it is a requirement of the C18 compiler that every section is located within a single bank. If there is a region that spans more than one bank, it is possible that the  section placed in this region will end up crossing a bank boundary. This causes the compiler generated variable access code to fail.  The following is an extract from the MPLAB linker manual: You must not combine data memory regions when using MPLINK Linker with MPLAB C18 C compiler. MPLAB C18 requires that any section be located within a single bank. As the freeRTOS linker script defines one large area crossing multiple banks, the linker will place sections within this area, without restraining them to separate banks. The default freeRTOS heap size is 1024 bytes long, leaving just two banks for variables.  I suppose it is therefore unlikely that a section will finish up crossing a bank boundary. I had reduced the heap size to make more room for global variables. Perhaps this is why this problem appears not to  have been experienced by others. My fix, which admittedly lacks elegance, is as follows: Mod the heap_1.c file to place the heap in a section named heap_section. I.e. #pragma udata heap_section static struct xRTOS_HEAP {     unsigned portLONG ulDummy;     unsigned portCHAR ucHeap[ portTOTAL_HEAP_SIZE ]; } xHeap; #pragma udata Change the linker script to: LIBPATH . FILES c018i.o FILES clib.lib FILES p18f252.lib CODEPAGE   NAME=vectors    START=0x0            END=0x39           PROTECTED CODEPAGE   NAME=page       START=0x3A           END=0x7DBF CODEPAGE   NAME=debug      START=0x7DC0         END=0x7FFF         PROTECTED CODEPAGE   NAME=idlocs     START=0x200000       END=0x200007       PROTECTED CODEPAGE   NAME=config     START=0x300000       END=0x30000D       PROTECTED CODEPAGE   NAME=devid      START=0x3FFFFE       END=0x3FFFFF       PROTECTED CODEPAGE   NAME=eedata     START=0xF00000       END=0xF000FF       PROTECTED ACCESSBANK NAME=accessram  START=0x0            END=0x7F DATABANK   NAME=gpr0       START=0x80           END=0xFF DATABANK   NAME=heapregion       START=0x100          END=0x3FF    PROTECTED DATABANK   NAME=gpr4       START=0x400          END=0x4FF DATABANK   NAME=gpr5       START=0x500          END=0x5F3 DATABANK   NAME=dbgspr     START=0x5F4          END=0x5FF          PROTECTED ACCESSBANK NAME=accesssfr  START=0xF80          END=0xFFF          PROTECTED SECTION    NAME=CONFIG     ROM=config SECTION    NAME=heap_section  RAM=heapregion STACK SIZE=0x60 RAM=gpr4 If the heap size exceeds the heap_region defined in the linker script, the linker will complain. However, it is possible to waste precious RAM by defining the heap to be significantly smaller than heap_region.  It is therefore advisable to ensure that the size of heap_region  is only slightly larger than the actual heap size. I would appreciate learning of a more elegant solution. As all heap allocated data is accessed via pointers, Im assuming that there is no bank problem here, but would like confirmation. Regards, John Franklin

Possible PIC port problem with linker script.

Thanks for the detailed feedback. This is something that needs consideration for V3?

Possible PIC port problem with linker script.

Hi John, I think my problem is also related to this limitation. In fact I think the problem is far more serious than what you describe. The linker and the C18 compiler assume that all RAM sections will fit inside a memory region. This means that 1) you cannot define more than 256 bytes of global/static variables inside one source file since 1 file will result in one section idata_SourceFileName.c. Maybe the compiler (not the linker) detects this and generates multiple datasections for 1 source file, but I’m not counting on that. 2) if the compiler is smart and generates multiple datasections for 1 source file: arrays still cannot be longer than 256 bytes, since the compiler will not set the BSR accordingly 3) I guess it is not possible with PIC to have a software stack of more than 256 bytes, so the number of local variables is also limited. How does this apply to FreeRTOS? The memory allocation function is not aware of the bank limitation of the PIC, thus it will just place a variable at the first available RAM position. your fix prevents global variables to be allocated inside the ‘bigchunk’ memory (where they can cross a bank boundary) and forces them in one of the banked sections. This also forces the start of the heap back to the start of a bank. (Before, the heap would be placed after some globals variables, isn’t it?) This is how I think your fix worked: the task.c variables are moved back down into the bank. However, this doesn’t fix problems inside the heap! I think 2 problems are to be expected. FreeRTOS should ensure that the software stack for each task is located inside 1 memory bank. semi-global variables such as Queues should be located inside 1 memory bank. Now, my problem is the following: I am using the PIC18F8525 to communicate with a magnet-card reader. The data from the RS232 is put in a queue at 38000baud, the tick is 1kHz, so I receive about 4 bytes per tick The queue is 80 bytes long. To read the queue, I wanted to use something like this: while ((cSerial2GetChar( RS232port, &cCardReaderReceivedByte, 10 )))        // wait for response {        // Get response     responsestring[CardReaderLocalIndex++]=cCardReaderReceivedByte; } Which doesn’t work. It just seems to exit the loop while there is still data in the queue. Maybe the task receives a resume signal during the waiting period? If only I knew from where. The alternative routine works fine. do {        // Get response     if (cSerial2GetChar( RS232port, &cCardReaderReceivedByte, 10 ))     {         responsestring[CardReaderLocalIndex++]=cCardReaderReceivedByte;     }     vTaskDelay(2); }while (ucQueueMessagesWaiting( xRS232RxedChars)>0 ); I do not understand why. The first routine should loop until there has been no data for 10 ticks. The second monitors the state of the queue after just 2 ticks and actually gets all the data. My impression for some weeks now has been that there is a memory problem. The queue array is 80 bytes long, so there is about 30% chance of a bank problem. Another problem is that semaphores sometimes do not seem to block other tasks. Since a semaphore is a queue, I think this problem is related to the first. After your posting I feel much stronger about this. I’d like to hear your (or Richard’s) answer to this. Thanks Paul van der Hulst

Possible PIC port problem with linker script.

Hi Paul. Ill try and address your points, as best I can. Ive indented your text with a few underscores. ___The linker and the C18 compiler assume that all RAM sections will fit inside a memory ___region. ___This means that ___1) you cannot define more than 256 bytes of global/static variables inside one source file ___since 1 file will result in one section idata_SourceFileName.c. Maybe the compiler (not ___the linker) detects this and generates multiple datasections for 1 source file, but I’m not ___counting on that. It is possible to create variables larger than a single bank. This is documented in a Microchip pdf file. You can get it from this link: http://ww1.microchip.com/downloads/en/DeviceDoc/51295E.pdf __2) if the compiler is smart and generates multiple data sections for 1 source file: arrays still ___cannot be longer than 256 bytes, since the compiler will not set the BSR accordingly Arrays which span multiple banks have to be accessed in a special way, see the above pdf. This raised some  concern about the way the heap was allocated in freeRTOS. However, Ive examined the compiler generated code and all seems fine. ___3) I guess it is not possible with PIC to have a software stack of more than 256 bytes, ___so the number of local variables is also limited. The normal C18 stack can be greater than 256 bytes, select the C18 large memory model. In this case the BSR is not used for stack access. Stacks in freeRTOS are allocated on the heap and accessed without using the BSR, so size should not present a problem. ___The memory allocation function is not aware of the bank limitation of the PIC, thus it will ___just place a variable at the first available RAM position. By default all global/static variables are placed in the section allocated to the source file that they are defined in. These will be mapped to one, and only one,  of the regions defined in the linker script. Provided each linker script defined region does not span a bank, all is well. The compiler generated code will set the BSR to the relevant bank. ___your fix prevents global variables to be allocated inside the ‘bigchunk’ memory (where ___they can cross a bank boundary) and forces them in one of the banked sections. This ___also forces the start of the heap back to the start of a bank. (Before, the heap would be ___placed after some globals variables, isn’t it?) This is how I think your fix worked: the ___task.c variables are moved back down into the bank. However, this doesn’t fix __problems inside the heap! Not sure I follow this. Its true that nothing other than the heap can now occupy the heap_region. If, for example,  your heap size is a lot less than heap_region, then the remaining space in heap region is wasted, the compiler will not use it for global variables. All global/static variables will be placed in one of the remaining linker script defined memory regions. You can examine the .map file to verify the placing of all variables, including the heap. __I think 2 problems are to be expected. __FreeRTOS should ensure that the software stack for each task is located inside 1 __memory bank. __semi-global variables such as Queues should be located inside 1 memory bank. As memory allocated on the heap, e.g. task stacks, queues etc, are accessed via pointers the compiler will not use the BSR. It is therefore irrelevant which bank they happen to be in. This certainly seems to be the case. Ive looked at the C18 generated code for signs that the BSR is being used to access the heap, all seems OK. I would, however, feel happier to have this confirmed. __The data from the RS232 is put in a queue at 38000baud, the tick is 1kHz, so I receive __about 4 bytes per tick __The queue is 80 bytes long. __To read the queue, I wanted to use something like this: __while ((cSerial2GetChar( RS232port, &cCardReaderReceivedByte, 10 ))) // wait for __response __{ // Get response __responsestring[CardReaderLocalIndex++]=cCardReaderReceivedByte; __} __Which doesn’t work. It just seems to exit the loop while there is still data in the queue. __Maybe the task receives a resume signal during the waiting period? If only I knew from __where. I had similar problems with an earlier version of the C18 compiler. The C18 code generated for pushing function parameters onto the stack was faulty. See my previous post. Changing to the latest version cured this, as could be seen by examining the compiler generated code. The behaviour resulting from variables crossing banks is many and varied, including erratic queue behaviour. I was at first mystified why small, apparently irrelevant, changes to my code could appear to fix the problem. I eventually traced this to the fact that the change had now pushed a variable, that was previously straddling two banks, into just one bank. This is of-course, the whole purpose of the changes to the linker script, to ensure that variables cannot cross a bank boundary. Just to re-iterate, variables which are allocated on the heap (e.g. queues), are not accessed by using the BSR, so can cross bank regions. At least, as far as I can ascertain, this seems to be the case. Have you made the changes I proposed in my post, and are still having problems? I hope this is of some use. Regards, John Franklin

Possible PIC port problem with linker script.

The memory allocation functions in the download are intended for demonstration and will not always be suitable. Does the pic compiler come with a malloc() library function?  If so then presumably it must take into account the banking issues and could be used in place of the heap2 function.  The heap3 function basically uses mallocs within critical regions.  This is fine provided you don’t want to malloc at real time/run time where the malloc() function would most likely take too long to complete.

Possible PIC port problem with linker script.

Hi John, As you pointed out: most of my points concerning banks and the BSR were wrong, I had forgotten all about indirect addressing… I spent most of the day stepping through the code and I now think the page thing is not an issue for data on the heap. If its worth anything: I also am pretty sure the C18 generated code is ok. By the way, I am using the newest versions of both FreeRTOS (2.6.1) and C18 (2.42) and have changed the code along the lines you proposed in your post. The heap is now the only data object in that section (and protected), all other sections span only 1 bank. Also my mathdata and .tmpdata are 0x14 and 0x04 long, so I changed post.c accordingly. Still, the queue readout and semaphore problems persists. Since taking a semaphore is the same (by a #define) as receiving something from a queue, this will be my next focus. To Mr. Nobody: malloc is not available by default. Microchip appears to have a simple SRAM memory manager in c  available, but this only allocates blocks up to 126 bytes. Since this does not seem to be the source of the problem right now, I will leave it at that. Greetings, Paul van der Hulst

Possible PIC port problem with linker script.

Hi Paul. Thanks for the feedback. I would be interested in knowing what you discover. Good luck. Regards, John Franklin

Possible PIC port problem with linker script.

Hi, I finally tracked down the problem. As you remember I had problems with serial i/o The situation: An ISR sends data to the queue (at 38000 baud) The communication task reads the data from the queue through the intermediate function cSerial2GetChar. Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE without having been blocked. I checked this by keeping track of the TickCount before entering the function and after returning. The problem lies in the generated assembly code. I included the C code and the assembly section below: -———————————————————————————- if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) ) {     return ( portCHAR ) pdTRUE; } 14291   6FA4  ECEE                             CALL cQueueReceive, 0                  14292   6FA6  F023                             NOP                                    14293   6FA8  6EF5                             MOVWF TABLAT, ACCESS                   14294   6FAA  0E06                             MOVLW 0x6                              14295   6FAC  5CE1                             SUBWF FSR1L, W, ACCESS                 14296   6FAE  E202                             BC 0x6fb4                              14297   6FB0  6AE1                             CLRF FSR1L, ACCESS                     14298   6FB2  52E5                             MOVF POSTDEC1, F, ACCESS               14299   6FB4  6EE1                             MOVWF FSR1L, ACCESS                    14300   6FB6  50F5                             MOVF TABLAT, W, ACCESS                 14301   6FB8  0900                             IORLW 0                                14302   6FBA  E003                             BZ 0x6fc2                              -———————————————————————————- After returning from the call, WREG is copied to TABLAT (line 14293) In line 14300 TABLAT is copied back to WREG and the result is tested (the ‘if’ condition). Apparently, with the high interruptfrequency I’m using, one of the interrupt functions is modifying TABLAT, whigh results in the wrong ‘if’ evaluation. I’m still trying to find out what the intermediate code is for, I have not been able to reproduce this. It probably has something to do with the function parameters being pointers or something like that. Functions which only return a ‘char’ value and have no or only one char parameter produce the code: call functionname,0 nop iorlw 1 bz 0xxxxx     <– destination address I fixed this by including TABLAT in the RS232 reception ISR function #pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata") void vSerial2RxISR( void ) { static portCHAR cChar,cWoken; Now communication is flawless. I guess I have to do the same thing with the other ISR functions. Greetings Paul van der Hulst

Possible PIC port problem with linker script.

Hi, I finally tracked down the problem. As you remember I had problems with serial i/o The situation: An ISR sends data to the queue (at 38000 baud) The communication task reads the data from the queue through the intermediate function cSerial2GetChar. Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE without having been blocked. I checked this by keeping track of the TickCount before entering the function and after returning. The problem lies in the generated assembly code. I included the C code and the assembly section below: -———————————————————————————- if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) ) {     return ( portCHAR ) pdTRUE; } 14291   6FA4  ECEE                             CALL cQueueReceive, 0                  14292   6FA6  F023                             NOP                                    14293   6FA8  6EF5                             MOVWF TABLAT, ACCESS                   14294   6FAA  0E06                             MOVLW 0x6                              14295   6FAC  5CE1                             SUBWF FSR1L, W, ACCESS                 14296   6FAE  E202                             BC 0x6fb4                              14297   6FB0  6AE1                             CLRF FSR1L, ACCESS                     14298   6FB2  52E5                             MOVF POSTDEC1, F, ACCESS               14299   6FB4  6EE1                             MOVWF FSR1L, ACCESS                    14300   6FB6  50F5                             MOVF TABLAT, W, ACCESS                 14301   6FB8  0900                             IORLW 0                                14302   6FBA  E003                             BZ 0x6fc2                              -———————————————————————————- After returning from the call, WREG is copied to TABLAT (line 14293) In line 14300 TABLAT is copied back to WREG and the result is tested (the ‘if’ condition). Apparently TABLAT is changed between returning from the call and the evaluation at line 14301. This can be either a task-switch or an ISR. In this case it is an ISR, ISRs don’t save TABLAT by default. I’m still trying to find out what the intermediate code is for, I have not been able to reproduce this. It probably has something to do with the function parameters being pointers or something like that. Functions which only return a ‘char’ value and have no or only one char parameter produce the code: call functionname,0 nop iorlw 1 bz 0xxxxx     <– destination address I fixed this by including TABLAT in the RS232 reception ISR function #pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata") void vSerial2RxISR( void ) { static portCHAR cChar,cWoken; Now communication is flawless. I guess I have to do the same thing with the other ISR functions. Greetings Paul van der Hulst

Possible PIC port problem with linker script.

Hi, I finally tracked down the problem. As you remember I had problems with serial i/o The situation: An ISR sends data to the queue (at 38000 baud) The communication task reads the data from the queue through the intermediate function cSerial2GetChar. Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE without having been blocked. I checked this by keeping track of the TickCount before entering the function and after returning. The problem lies in the generated assembly code. I included the C code and the assembly section below: -———————————————————————————- if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) ) {     return ( portCHAR ) pdTRUE; } 14291   6FA4  ECEE                             CALL cQueueReceive, 0                  14292   6FA6  F023                             NOP                                    14293   6FA8  6EF5                             MOVWF TABLAT, ACCESS                   14294   6FAA  0E06                             MOVLW 0x6                              14295   6FAC  5CE1                             SUBWF FSR1L, W, ACCESS                 14296   6FAE  E202                             BC 0x6fb4                              14297   6FB0  6AE1                             CLRF FSR1L, ACCESS                     14298   6FB2  52E5                             MOVF POSTDEC1, F, ACCESS               14299   6FB4  6EE1                             MOVWF FSR1L, ACCESS                    14300   6FB6  50F5                             MOVF TABLAT, W, ACCESS                 14301   6FB8  0900                             IORLW 0                                14302   6FBA  E003                             BZ 0x6fc2                              -———————————————————————————- After returning from the call, WREG is copied to TABLAT (line 14293) In line 14300 TABLAT is copied back to WREG and the result is tested (the ‘if’ condition). Apparently TABLAT is changed between returning from the call and the evaluation at line 14301. This can be either a task-switch or an ISR. In this case it is an ISR, ISRs don’t save TABLAT by default. I’m still trying to find out what the intermediate code is for, I have not been able to reproduce this. It probably has something to do with the function parameters being pointers or something like that. Functions which only return a ‘char’ value and have no or only one char parameter produce the code: call functionname,0 nop iorlw 1 bz 0xxxxx     <– destination address I fixed this by including TABLAT in the RS232 reception ISR function #pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata") void vSerial2RxISR( void ) { static portCHAR cChar,cWoken; Now communication is flawless. I guess I have to do the same thing with the other ISR functions. Greetings Paul van der Hulst

Possible PIC port problem with linker script.

Hi, I finally tracked down the problem. As you remember I had problems with serial i/o The situation: An ISR sends data to the queue (at 38000 baud) The communication task reads the data from the queue through the intermediate function cSerial2GetChar. Although there is a block time of 10 ticks, sometimes the function seems to return pdFALSE without having been blocked. I checked this by keeping track of the TickCount before entering the function and after returning. The problem lies in the generated assembly code. I included the C code and the assembly section below: -———————————————————————————- if( cQueueReceive( xRS232RxedChars, pcRxedChar, xBlockTime ) ) {     return ( portCHAR ) pdTRUE; } 14291   6FA4  ECEE                             CALL cQueueReceive, 0                  14292   6FA6  F023                             NOP                                    14293   6FA8  6EF5                             MOVWF TABLAT, ACCESS                   14294   6FAA  0E06                             MOVLW 0x6                              14295   6FAC  5CE1                             SUBWF FSR1L, W, ACCESS                 14296   6FAE  E202                             BC 0x6fb4                              14297   6FB0  6AE1                             CLRF FSR1L, ACCESS                     14298   6FB2  52E5                             MOVF POSTDEC1, F, ACCESS               14299   6FB4  6EE1                             MOVWF FSR1L, ACCESS                    14300   6FB6  50F5                             MOVF TABLAT, W, ACCESS                 14301   6FB8  0900                             IORLW 0                                14302   6FBA  E003                             BZ 0x6fc2                              -———————————————————————————- After returning from the call, WREG is copied to TABLAT (line 14293) In line 14300 TABLAT is copied back to WREG and the result is tested (the ‘if’ condition). Apparently TABLAT is changed between returning from the call and the evaluation at line 14301. This can be either a task-switch or an ISR. In this case it is an ISR, ISRs don’t save TABLAT by default. I’m still trying to find out what the intermediate code is for, I have not been able to reproduce this. It probably has something to do with the function parameters being pointers or something like that. Functions which only return a ‘char’ value and have no or only one char parameter produce the code: call functionname,0 nop iorlw 1 bz 0xxxxx     <– destination address I fixed this by including TABLAT in the RS232 reception ISR function #pragma interruptlow vSerial2RxISR save=PRODH, PRODL,TABLAT, section(".tmpdata") void vSerial2RxISR( void ) { static portCHAR cChar,cWoken; Now communication is flawless. I guess I have to do the same thing with the other ISR functions. Greetings Paul van der Hulst

Possible PIC port problem with linker script.

Well, sorry about double posting, remove some if possible… I posted the message but is wouldn’t display. Again, sorry Paul van der Hulst

Possible PIC port problem with linker script.

I am developing a project for LPC2103. For this I took ARM7_LPC2106_GCC demo as base. Till now I am using heap2.c and my code is working fine. Now, I want to use heap3.c. For this , in Linker script I setup the heap after bss section like,     .heap :     {        __heap_start__ = .;        *(.heap)        . += 2k;        __heap_end__ = .;     } >ram   But my code is not is not working. Do I have to do something more than this?