POV-Ray : Newsgroups : povray.bugreports : alpha.10064268 outputting strings Server Time
8 Nov 2024 21:07:41 EST (-0500)
  alpha.10064268 outputting strings (Message 1 to 7 of 7)  
From: jr
Subject: alpha.10064268 outputting strings
Date: 16 Jun 2020 10:10:01
Message: <web.5ee8d25356643f3f4d00143e0@news.povray.org>
hi,

thought I had read that strings, in 3.8, are no longer limited to 256 characters
length, however, cannot find that reference now.

the (edited) output below is debug info about a quadtree.  I build the lists of
points in the nodes' "buckets", for display, in a string with '#local S =
concat(S,...);'.

the list for node #22 is truncated, and the newline "swallowed", whereas node
#35's list is printed in full.

the tree has 100+ nodes and truncation occurs for more than half of nodes (with
points).

so what is the status of strings in 3.8?  does the 256 char limit still apply?



Persistence of Vision(tm) Ray Tracer Version 3.8.0-alpha.10064268.unofficial
....
==== [Parsing...] ==========================================================
-----[quadtree dump]----------------------------------------------------
          root node id: 1
             area from: <-5.0000, -5.0000>
                    to: <5.0000, 5.0000>
               # nodes: 125
           bucket size: 10
              # points: 500
....
                bucket: <-3.8206, -4.8561, 0.0000>
   <-4.4137, -3.8171, 0.0000>
   <-4.8114, -4.8691, 0.0000>
   <-4.1970, -4.0342, 0.0000>
   <-4.7922, -4...
               node id: 23
....
                bucket: <1.3956, -4.6061, 0.0000>
   <1.7214, -4.7567, 0.0000>
   <2.3263, -4.5134, 0.0000>
   <1.8115, -4.9289, 0.0000>
   <2.0349, -4.3726, 0.0000>
   <2.4509, -4.1583, 0.0000>

               node id: 36
....



(also posting a couple of search results from the same tree in p.b.image)


regards, jr.


Post a reply to this message

From: B  Gimeno
Subject: Re: alpha.10064268 outputting strings
Date: 17 Jun 2020 12:15:00
Message: <web.5eea40cf71bac6e7a723b8610@news.povray.org>
Here is the reference to the old 256 limit in v3.6

https://www.povray.org/documentation/view/3.6.1/510/
STRING_LITERAL STRING_LITERAL:
    "up to 256 ASCII characters"


And the link to the updated 3.8 version:
http://wiki.povray.org/content/Reference:Strings#String_Functions
STRING:
  STRING_FUNCTION   |
  STRING_IDENTIFIER |
  STRING_LITERAL STRING_LITERAL:
  "up to 256 ASCII characters"

I propose to use as many concat() as necessary, if that was the problem.


By the way, has anyone managed to write unicode characters (\unnnn) above the
ascii 128 limit to a text file? I would like to play with the rest of the ascii
characters but I only get blank spaces; Yes, I have already tried utf-8 in
general settings, but I do not intend to generate a text object in an image
using a truetype font or an ANSI extended font, but a separate plain text file
whose content is dumped by
Pov-Ray.

Regards
BGimeno


Post a reply to this message

From: jr
Subject: Re: alpha.10064268 outputting strings
Date: 17 Jun 2020 14:15:01
Message: <web.5eea5d6671bac6e74d00143e0@news.povray.org>
hi,

"B. Gimeno" <nomail@nomail> wrote:
> Here is the reference to the old 256 limit in v3.6
> ...
> And the link to the updated 3.8 version:
> ...
>   "up to 256 ASCII characters"

thanks.  no idea why I misremembered.


> I propose to use as many concat() as necessary, if that was the problem.

yes, I think that will be the only option, now.


> By the way, has anyone managed to write unicode characters (\unnnn) above the
> ascii 128 limit to a text file? I would like to play with the rest of the ascii
> characters but I only get blank spaces; Yes, I have already tried utf-8 in
> general settings, but I do not intend to generate a text object in an image
> using a truetype font or an ANSI extended font, but a separate plain text file
> whose content is dumped by
> Pov-Ray.

if the '\Unnnn' notation does not work, perhaps can you output (byte-wise) UTF8
code points?  <https://www.utf8-chartable.de/>


regards, jr.

(natty macro, btw.  will try it later on.  thanks)


Post a reply to this message

From: B  Gimeno
Subject: Re: alpha.10064268 outputting strings
Date: 17 Jun 2020 17:45:06
Message: <web.5eea8e9671bac6e7a723b8610@news.povray.org>
"jr" <cre### [at] gmailcom> wrote:
> hi,
> > By the way, has anyone managed to write unicode characters (\unnnn) above the
> > ascii 128 limit to a text file? I would like to play with the rest of the ascii
> > characters but I only get blank spaces; Yes, I have already tried utf-8 in
> > general settings, but I do not intend to generate a text object in an image
> > using a truetype font or an ANSI extended font, but a separate plain text file
> > whose content is dumped by Pov-Ray.
>
> if the '\Unnnn' notation does not work, perhaps can you output (byte-wise) UTF8
> code points?  <https://www.utf8-chartable.de/>

Not sure of anything after reading this:
http://news.povray.org/povray.beta-test/message/%3C5c2e6746%40news.povray.org%3E/#%3C5c2e6746%40news.povray.org%3E

....and that someone was inquirinq in a similar way 20 years ago.
http://news.povray.org/povray.beta-test/thread/%3C3BADC84E.1AB6A6C0%40post8.tele.dk%3E/?mtop=176162


onemiGB


Post a reply to this message

From: William F Pokorny
Subject: Re: alpha.10064268 outputting strings
Date: 17 Jun 2020 19:50:19
Message: <5eeaac3b$1@news.povray.org>
On 6/17/20 5:43 PM, B. Gimeno wrote:
...
> 
> Not sure of anything after reading this:
>
http://news.povray.org/povray.beta-test/message/%3C5c2e6746%40news.povray.org%3E/#%3C5c2e6746%40news.povray.org%3E
> 
> ....and that someone was inquirinq in a similar way 20 years ago.
>
http://news.povray.org/povray.beta-test/thread/%3C3BADC84E.1AB6A6C0%40post8.tele.dk%3E/?mtop=176162
> 

Thanks for those links. I have no recollection of either thread...

Bill P.


Post a reply to this message

From: Thorsten
Subject: Re: alpha.10064268 outputting strings
Date: 18 Jun 2020 02:27:01
Message: <5eeb0935$1@news.povray.org>
On 18.06.2020 01:50, William F Pokorny wrote:
> On 6/17/20 5:43 PM, B. Gimeno wrote:
> ...
>>
>> Not sure of anything after reading this:
>>
http://news.povray.org/povray.beta-test/message/%3C5c2e6746%40news.povray.org%3E/#%3C5c2e6746%40news.povray.org%3E

>>
>> ....and that someone was inquirinq in a similar way 20 years ago.
>>
http://news.povray.org/povray.beta-test/thread/%3C3BADC84E.1AB6A6C0%40post8.tele.dk%3E/?mtop=176162

> 
> Thanks for those links. I have no recollection of either thread...

Well, I do ;-)

To answer some of the questions, assuming nothing has been changed in 
the meantime ...

Practically, there are four uses of strings in POV-Ray. One is for TTF 
output. Here, UTF-8 cannot be used internally, but UCS2/4 is used, and 
the strings are converted for this use. The reason is that UTF-8 is a 
variable length encoding (as is UTF-16), which is not very efficient to 
decode on the fly, though when building the extrusions for text objects, 
that would hardly matter, but anyway, that is the way it is.

The other main use is for include files and all other files. Here a path 
is needed. 20 years back, opening anything but ASCII based paths was a 
mess, and to an extend it still is. There was some abstraction put in 
place, and by all means, here Unicode or more precisely the charset, 
should also work these days.

The next use case is for debug output.This output goes to the console or 
message window in a GUI. Originally, neither supported Unicode (20 years 
ago), so there comes the implementation: Replace any character code => 
128 with a space. This eliminates the need to support Unicode or UTF-8 
in those windows and on terminals.

And this brings us to the similar use: The output of text files. They 
used to use the same implementation. And from what I am reading, i guess 
nothing has changed.

Oh, and for the 256 character limit, that comes from yet another source: 
The tokenizer responsible for parsing strings reuses the same buffer as 
it uses for parsing tokens. So if this hasn't been changed, that is 
where the limit comes from.

In short, as you can see, the string implementation touches many parts 
of POV-Ray, and in order to make much progress, all these points would 
have to be fixed simultaneously.

Thorsten


Post a reply to this message

From: William F Pokorny
Subject: Re: alpha.10064268 outputting strings
Date: 18 Jun 2020 09:16:52
Message: <5eeb6944$1@news.povray.org>
On 6/18/20 2:27 AM, Thorsten wrote:
> On 18.06.2020 01:50, William F Pokorny wrote:
...
>>
>> Thanks for those links. I have no recollection of either thread...
> 
...
> 
> In short, as you can see, the string implementation touches many parts 
> of POV-Ray, and in order to make much progress, all these points would 
> have to be fixed simultaneously.
> 

Thank you for overview. :-)

Bill P.


Post a reply to this message

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.