{Array|Linked}List: Proposal of two new C'tors and methods

Hello World!

I would like to hear your opinion about the addition of two new constructors

and two new methods for the concrete implementations of the j.u.List interface

(ArrayList and LinkedList). I show them for the AbstractList class:

First the specifications of the C'tors:

/**

* Creates a new List from the elements of an iterable object.

* The sequence order in the resulting list equals

* the order defined by the argument's iterator.

*/

public AbstractList(Iterable<? extends T> iterable);

/**

* Creates a new List from the elements represented by an iterator.

* The sequence order in the resulting list equals

* the order defined by the argument iterator.

*/

public AbstractList(Iterator<? extends T> iterator);

And here are the methods (I leaf out the obvious comments):

public void addAll(Iterable<? extends T> iterable);

public void addAll(Iterator<? extends T> iterator);

Reasoning:

In Java 1.5 (and in the upcoming 1.6, too), we only have a C'tor for creating a list

from a j.u.Collection. Until 1.4 this was ok, but with the introduction of the

Iterable interface, together with the for-each loop, the definition of iterators

for user defined classes becomes a common task. So this should also be recognized in

the definition of the standard APIs. Currently I have to define utility functions

for this purpose in all my projects. While this is very easy to do (see below),

I feel this is not the right thing to doo -- it should be there out of the box.

The first C'tor and addAll method, which gets an Iterable object as its argument,

is a real generalization of the existing Collection's C'tor and addAll method,

because Collection is a subinterface of Iterable. So this won't break any code,

even if the old C'tor would be replaced by the new. But for efficiency reasons

(no runtime type check), I would keep the Collection C'tor -- this is

luckily possible in Java, because the compiler always uses the most specific

type definition when selecting an overloaded method (here Collection is

more specific than Iterable).

Notice that one could think about pulling up my proposed methods to

all CollectionS. But one has to consider a few things:

* In SetS, only one of the argument's elements will be inserted in the result.

* For non SortedSets, the order of the resulting objects will be unknown.

I think, this would not be a problem.

Here is a straightforward implementation of the C'tors and addAll methods:

public AbstractList(Iterable<? extends T> iterable) {

this(iterable.iterator());

}

public AbstractList(Iterator<? extends T> iterator) {

super();

this.addAll(iterator);

}

private void addAll(Iterable<? extends T> iterable) {

this.addAll(iterable.iterator());

}

private void addAll(Iterator<? extends T> iterator) {

while (iterator.hasNext()) {

this.add(iterator.next());

}

}

Greetings,

Michael

[3237 byte] By [m_schneia] at [2007-9-24]
# 1

> Hello World!

>

> I would like to hear your opinion

After reading your post once, I don't object and even tend to agree that it's a good idea to include this functionality in the standard API, rather than having everyone write utility functions for it.

I've been thinking about the same issue and how there are many methods accepting Collection which could really be accepting Iterable. I'm less sure about the variations with Iterator but they have their merit as well.

So IMHO Your proposal has merit and is well written; I'd say submit as RFE and you have my vote.

Lokoa at 2007-7-13 > top of java,Core,Core APIs...
# 2

Good post, and i agree although i think only the Iterable versions have merit. The sole reason Iterator classes exist is to iterate over Collection classes, and if you have a Collection you can just pass that to the existing constructors. So

new MyCollection(someCollection)

rather than

new MyCollection(someCollection.iterator())

which is basically what you'd be doing.

Iterable on the other hand is defined as an interface that allows custom objects to be used as a target for for-each loops, and although that still requires an Iterator internally it's more weakly linked to the JCF in my opinion and thus a better candidate for the constructors you propose.

Nevertheless i agree with your post, have my vote.

remonvva at 2007-7-13 > top of java,Core,Core APIs...
# 3
If I may make a guess, I think the OP wants the Iterator version for cases where you don't want the entire Iterable, but only the elements from a given iterator.
Lokoa at 2007-7-13 > top of java,Core,Core APIs...
# 4

> The sole reason Iterator classes exist is to iterate over Collection classes,

Actually, no. It's possible to write an Iterator for anything you like. I have written a ResultSetIterator, for example. You could write a BufferedReaderIterator that returned lines from a text file. No Collections necessary.

DrClapa at 2007-7-13 > top of java,Core,Core APIs...
# 5

> > The sole reason Iterator classes exist is to

> iterate over Collection classes,

>

> Actually, no. It's possible to write an Iterator for

> anything you like. I have written a

> ResultSetIterator, for example.

As another example, take the JENA Semantic Web Framework

<http://jena.sourceforge.net>,

which defines several own iterators in its classes.

You get them as a result of a query operation,

e.g. on an RDF Model (Model.listStatements()).

But because JENA is a before-J1.5 development, its classes

do not implement the Iterable interface. And I won't expect

JENA to switch to 1.5, because it is an important base

for a lot of existing code.

And, BTW, Iterators are a "Design Pattern" according to Gamma, et.a.,

so one would expect them to occure on many other places

than only in the CollectionS framework.

Michael

m_schneia at 2007-7-13 > top of java,Core,Core APIs...
# 6

> > Hello World!

> > I would like to hear your opinion

>

> So IMHO Your proposal has merit and is well written;

> I'd say submit as RFE and you have my vote.

Thanks! :) So, after getting some first positive answers, I am going to submit this as an RFE. But, Loko (or someone else here), can you please tell me, *how* to submit an RFE?

Bye,

Michael

m_schneia at 2007-7-13 > top of java,Core,Core APIs...
# 7
> Thanks! :) So, after getting some first positive> answers, I am going to submit this as an RFE. But,> Loko (or someone else here), can you please tell me,> *how* to submit an RFE? http://bugs.sun.com/services/bugreport/index.jsp
warnerjaa at 2007-7-13 > top of java,Core,Core APIs...
# 8
Please do post back here with a link once you have submitted it.
Lokoa at 2007-7-13 > top of java,Core,Core APIs...
# 9

> > The sole reason Iterator classes exist is to

> iterate over Collection classes,

>

> Actually, no. It's possible to write an Iterator for

> anything you like. I have written a

> ResultSetIterator, for example. You could write a

> BufferedReaderIterator that returned lines from a

> text file. No Collections necessary.

Yes i'm perfectly aware that an Iterator class (as opposed to the pattern)doesnt necessarily have to have a java.util.Collection as backing. However, even if your object provides a non-standard Iterator object it should probably do so through implementing the Iterable interface (and thus, Iterable.iterator()) rather than providing an Iterator object through some other means.

In the case if your BufferedReaderIterator you'd have two ways of doing it :

1) Implement a class implementing Iterator that accepts a BufferedReader as a constructor parameter (or setter etc.). Works fine, you can easily follow the Iterator contract.

2) Implement a class that encapsulates BufferedReader and provides an Iterator object through a method (typically .iterator()). Works fine, you'd provide a similar if not the same Iterator implementation to that proposed in option 1.

The thing is though, option 2 gives you reasonably useful encapsulation of the collection to iterate over (which can possible change, especially if it's a class you created yourself) as well as make your new collection a viable target for for-each loops :

for(String line: MyBufferedReader)

System.out.println(line);

I'd personally find that a better approach but i guess it's a matter of style, and it's entirely possible i'm forgetting about situations where this is impossible or just bad design.

Anyway, the poster above that pointed out you may want to pass an Iterator that iterates over a specific section of a collection or group of objects has a valid point, so i stand corrected either way ;)

remonvva at 2007-7-13 > top of java,Core,Core APIs...
# 10

> However, even

> if your object provides a non-standard Iterator

> object it should probably do so through implementing

> the Iterable interface (and thus,

> Iterable.iterator()) rather than providing an

> Iterator object through some other means.

Fair enough. Iterable is one of those new Java 5 things that I'm not up to speed on, clearly.

DrClapa at 2007-7-13 > top of java,Core,Core APIs...
# 11

@warnerja: Thank you for the link.

@Loko: Yes, I will tell to the thread, when I have submitted the request.

In the meanwhile I found that there is somewhat more to consider.

First, I now believe that the addAll methods should be part

of the Collection interface, not only of the List interface.

Currently Collection.addAll gets itself a Collection,

and this just means "all the elements of that collection".

The interface definition of Collection does not even say

anything about the order of the elements,

while the definition of List declares the order to be that of

the Collection's iterator() (the Iterable.iterator() !),

and the definition of Set says, that an element is only added,

if it is not already there. These kinds of specifications will

be trivially transfered to the Iterator/Iterable case.

Second: I think it would be consequent to also add

Iterable/Iterator versions of the following Collection methods

* containsAll

* removeAll

* retainAll

They all take CollectionS in their current version, and for each

of them I cannot see a reason, why they should be restricted

to CollectionS.

Third: There seem to be further special versions of those methods,

at least I have found

List<T>.addAll(int index, Collection<? extends T> c)

So I want to first go through the complete Collection framework

to see if there are other such methods.

I will work this out till the end of the week. After submission,

you will get noticed.

Thanks to all for the discussion!

Michael

m_schneia at 2007-7-13 > top of java,Core,Core APIs...
# 12

I would agree with you regarding all of your proposed additions; it certainly seems like the best route to take.

However, backward compatibility bites us in the leg once again. If you add those methods to Collection, my current Collection implementations are no longer complete; that is to say that my current 1.5 implementation of Bag will not compile using your new Collection interface. You can add constructors as much as you like, but adding methods to interfaces gets hairy.

Honestly, I'd like to see some new Java (2.0? Java3?) that addresses these sorts of problems by breaking backward compatibility but Sun is already worried about forking as a result of going open source. And that's a discussion for another thread... :)

Cheers!

tvynra at 2007-7-13 > top of java,Core,Core APIs...
# 13

tvynr wrote:

> If you add those methods to

> Collection, my current Collection implementations are

> no longer complete; that is to say that my current

> 1.5 implementation of Bag will not compile using your

> new Collection interface.

This depends: If there are reasonable implementations of those methods in the abstract base classes (like AbstractList), then there won't be a problem, as long as you subclass those abstract bases. And as I argued in my original posting, the implementations of the methods are rather straightforward. Of course, if you create your implementations from scratch (and there might be good reasons for doing this), then you are on your own again...

Michael

m_schneia at 2007-7-13 > top of java,Core,Core APIs...
# 14
I just want to inform you, that I have now submitted an RFE for adding the proposed methods and constructors. Currently the RFE is not yet visable in the bug database. I will tell you when I get mail from the reviewers. They told me that this might last three weeks.Michael
m_schneia at 2007-7-13 > top of java,Core,Core APIs...
# 15

Sounds like the usual process, yep.

I personally agree with you that it would be relatively simplistic for my HashBag implementation (which implements Bag, which extends Collection) to provide the methods in question. Unfortunately, it would still require me to modify older code to meet the expectations of the new interface. Additionally, since I must refer to things which were added in 1.5 (i.e., Iterable), I am now required to have a post-1.5 version of the code (implementing the new Collection interface) and a pre-1.5 version of the code (not doing so because Iterable does not exist in pre-1.5).

Sun's policy on this seems to be perfectly reasonable to me: nothing is ever added in such a way that code written for an older JVM (back to 1.2, anyway) won't compile in a newer one. In my case, it's just a couple classes. In a larger production environment, weeks of work might be involved in porting the code (and lots more work will be involved in maintaining two versions if the developers require that the code run on pre-new-Collection interface JVMs). While the way it's done at the moment isn't IMHO the best for new code, it does make old code continue to work.

I've personally never been in a situation where that sort of thing is a big deal, but I understand that it's reason enough for some people to pick a different language... and I think I understand why.

Mind you, I really like your suggestions. :) And, once again, if it would be possible to have a new Java which broke backward compatibility but had all those nice things that we keep wishing we could add to the interfaces and so on, I'd be among the first in the wagon. Thanks for submitting the RFE; I look forward to voting for it. :)

Cheers!

tvynra at 2007-7-21 > top of java,Core,Core APIs...
# 16

> I would agree with you regarding all of your proposed

> additions; it certainly seems like the best route to

> take.

>

> However, backward compatibility bites us in

> the leg once again. If you add those methods to

> Collection, my current Collection implementations are

> no longer complete; that is to say that my current

> 1.5 implementation of Bag will not compile using your

> new Collection interface. You can add constructors

> as much as you like, but adding methods to interfaces

> gets hairy.

Yes, but it isn't a new subject. Every new version of JDBC requires that previous drivers do something if they want to be recompiled.

The solution is either update it or don't compile it. And that is what the JDBC drivers do.

>

> Honestly, I'd like to see some new Java (2.0?

> Java3?) that addresses these sorts of problems by

> breaking backward compatibility but Sun is already

> worried about forking as a result of going open

> source. And that's a discussion for another

> thread... :)

Open source? Where do they say that?

jschella at 2007-7-21 > top of java,Core,Core APIs...
# 17

> Yes, but it isn't a new subject. Every new version

> of JDBC requires that previous drivers do something

> if they want to be recompiled.

>

> The solution is either update it or don't compile it.

> And that is what the JDBC drivers do.

Really? I didn't realize that they didn't keep JDBC backwards compatible. But I'd imagine breaking that in something like Collection or other core interfaces would make bigger waves.

I'd be all in favor of those new methods, but I'm sure there are other peopl (at least, I think there are... none of them seem to be posting in this thread...) who would have much larger issues with the idea.

> > Honestly, I'd like to see some new Java (2.0?

> > Java3?) that addresses these sorts of problems by

> > breaking backward compatibility but Sun is already

> > worried about forking as a result of going open

> > source. And that's a discussion for another

> > thread... :)

>

> Open source? Where do they say that?

http://www.vnunet.com/vnunet/news/2156205/sun-promises-open-source-java

http://news.com.com/Sun+promises+to+open-source+Java/2100-7344_3-6072760.html

Of course, I can't guarantee that this isn't an elaborate hoax... but it seems fairly reliable to me. :)

Cheers! ;)

tvynra at 2007-7-21 > top of java,Core,Core APIs...
# 18

>

> > > Honestly, I'd like to see some new Java (2.0?

> > > Java3?) that addresses these sorts of problems by

> > > breaking backward compatibility but Sun is already

> > > worried about forking as a result of going open

> > > source. And that's a discussion for another

> > > thread... :)

> >

> > Open source? Where do they say that?

>

> http://www.vnunet.com/vnunet/news/2156205/sun-promises

> -open-source-java

> http://news.com.com/Sun+promises+to+open-source+Java/2

> 100-7344_3-6072760.html

>

> Of course, I can't guarantee that this isn't an

> elaborate hoax... but it seems fairly reliable to me.

Interesting. Of course years ago they claimed that they would put it under a standards committee as well and backed out because they would have lost control of it (which doesn't seem to have gone away entirely.)

jschella at 2007-7-21 > top of java,Core,Core APIs...
# 19

> Interesting. Of course years ago they claimed that

> they would put it under a standards committee as well

> and backed out because they would have lost control

> of it (which doesn't seem to have gone away entirely.)

Hence elaborate hoax. :-D But we can hope. I look forward to being able to install the JRE on my Debian system via a package. :)

Cheers!

tvynra at 2007-7-21 > top of java,Core,Core APIs...
# 20

Hello again!

I am the original poster. My RFE is now in the database (6431636):

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6431636

It got some critics from the evaluation team, and those critics are more or less the same as discussed here in this thread, most notably by tvynr. It's all about backward compatibility. I tought about those points again and changed my mind, and with my mind I changed my RFE, and gave it a more conservative direction. Perhaps I should create a completely new RFE, but first let's see how this one develops.

So if you are still interested, please do not only read my original RFE (which will "probably not be accepted"), but also the critics of the eval team and my comments to them, which are appended to the RFE. Only then make your vote.

Thanks again to all for discussion,

Michael

m_schneia at 2007-7-21 > top of java,Core,Core APIs...
# 21

Read & voted.

I suppose every proposal regarding the Collections API will meet with the same "yeah great idea but we didn't think of it at first and now we can't do it... - and besides it's a very hard topic" response... which is understandable of course.

It's all water under the bridge now, but I've always considered the Collections API a bit odd in history. I believe other languages (like C++, upon which Java was partly based) had already shown at the time what great advantages can be had from separating collections, iterators and algorithms.

I find it weird that Java Collections have always had Iterators, and yet all algorithms are hardwired to use Collection. This includes the construction of one collection from others (the copy ctors) and the add/contains/removeAll as in Michael's proposal, but also some methods in class Collections. Take Collections.max(...), why was it originally conceived to operate on a Collection, when it naturally could operate on any sequence of objects, and interface Iterator already existed at the time? I suppose time constraints played a major role there.

Anyways... I still think this proposal is a good start towards a higher level of abstraction and a more flexible API of collections-iterators-algorithms. So I read and voted - and regained the urge to ramble about this subject :-)

Lokoa at 2007-7-21 > top of java,Core,Core APIs...